arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2607.06015cs.SD
收藏

我关心的音乐:对(几乎)任何音乐的大语言模型音乐感知技能进行自动化多模态基准测试

Music I Care About: Automated Multimodal Benchmarking of LLM Music Perception Skills on (Almost) Any Music

Charles University(查尔斯大学);Brno University of Technology(布拉格技术大学)

Tomáš Sourada, Katia Vendrame, Jan Hajič

AI总结:

针对当前音乐基准测试的局限,引入MusICA-MetaBench框架,利用结构化符号表示和预定义模板生成按需基准测试,通过ChoraleBricks数据集展示并确定规模,经与基线比较证明能测音乐感知,推动了大语言模型音乐感知跨模态评估。

摘要:

音乐是人类文化的基石,以多种数字形式存在。尽管多模态大语言模型有所发展,但当前音乐基准测试存在三大局限:大型静态基准测试评估资源密集且结果难以迁移;声称测量“音乐理解”的基准测试往往未要求音乐感知;不支持跨音乐模态的系统性能比较。为克服这些问题,我们引入了“我关心的音乐元基准测试”(MusICA-MetaBench)框架,它能根据用户提供的数据自动生成按需基准测试。通过利用结构化符号表示和预定义问题模板,构建探测音乐感知能力的问答对。我们用ChoraleBricks数据集展示该框架,确定基准测试规模,通过与仅文本和白噪声基线比较,证明问题能测量音乐感知。MusICA-MetaBench代表了大语言模型音乐感知跨模态评估的重大进步,提出了特定数据集的基准测试范式,实现了对音乐感知能力的高效按需评估。

英文摘要:

Music represents a cornerstone of human culture, existing digitally across diverse modalities, including audio, symbolic encodings (e.g., MIDI, MusicXML), and sheet music. Despite the advancement of Multimodal Large Language Models (MLLMs), current music benchmarks face three major limitations. First, large static benchmarks are resource-intensive to evaluate, and it remains unclear how their results transfer to diverse kinds of music beyond those included in the benchmark. Second, benchmarks claiming to measure "music understanding" often fail to require music perception. Third, they do not support systematic performance comparisons across musical modalities. To overcome these issues, we introduce the Music I Care About Meta-Benchmark (MusICA-MetaBench), a framework that automatically derives on-demand benchmarks directly from user-provided data. By leveraging structured symbolic representations (e.g., MusicXML) and our pre-defined question templates, we build multiple-choice question-answer pairs that probe music perception competencies, aligned with music pedagogy, across audio, music notation images, and symbolic files. We demonstrate our framework with the ChoraleBricks dataset, and experimentally determine benchmark sizes that ensure statistically reliable model comparisons for this setup. By comparing against text-only and white-noise baselines, we show our questions do measure music perception. Ultimately, MusICA-MetaBench represents a significant advancement in the cross-modal assessment of music perception for MLLMs. By proposing a dataset-specific benchmarking paradigm, it enables efficient on-demand evaluation of music perception capabilities.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑