IncompeBench: 一个宽松许可、细粒度的音乐信息检索基准
IncompeBench: A Permissively Licensed, Fine-Grained Benchmark for Music Information Retrieval
AI总结：
IncompeBench提供了一个高质量、宽松许可的音乐信息检索基准，包含大量标注数据和多样化查询，用于评估和改进音乐检索性能。
摘要：
多模态信息检索近年来取得了显著进展，利用深度预训练模型日益强大的多模态能力来跨模态表示信息。特别是音乐信息检索（MIR）的质量有了显著提升，神经表示的音乐甚至进入了日常消费品。然而，缺乏高质量的基准来评估音乐检索性能。为了解决这个问题，我们引入了IncompeBench，一个精心标注的基准，包含1,574个宽松许可、高质量的音乐片段，500个多样查询和超过125,000个个体相关性判断。这些标注是通过多阶段流程创建的，导致人类标注者和生成数据之间的一致性很高。所得到的数据集在https://huggingface.co/datasets/mixedbread-ai/incompebench-strict和https://huggingface.co/datasets/mixedbread-ai/incompebench-lenient公开可用，提示可在https://github.com/mixedbread-ai/incompebench-programs中找到。
英文摘要：
Multimodal Information Retrieval has made significant progress in recent years, leveraging the increasingly strong multimodal abilities of deep pre-trained models to represent information across modalities. Music Information Retrieval (MIR), in particular, has considerably increased in quality, with neural representations of music even making its way into everyday life products. However, there is a lack of high-quality benchmarks for evaluating music retrieval performance. To address this issue, we introduce \textbf{IncompeBench}, a carefully annotated benchmark comprising $1,574$ permissively licensed, high-quality music snippets, $500$ diverse queries, and over $125,000$ individual relevance judgements. These annotations were created through the use of a multi-stage pipeline, resulting in high agreement between human annotators and the generated data. The resulting datasets are publicly available at https://huggingface.co/datasets/mixedbread-ai/incompebench-strict and https://huggingface.co/datasets/mixedbread-ai/incompebench-lenient with the prompts available at https://github.com/mixedbread-ai/incompebench-programs.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
