arXiv 2509.040592025-09-26cs.CL

迈向AI音乐家：为音乐推理合成乐谱问题

Towards an AI Musician: Synthesizing Sheet Music Problems for Musical Reasoning

University of Science and Technology of China（中国科学技术大学）；Shanghai AI Laboratory（上海人工智能实验室）；Sichuan University（四川大学）；Shanghai Jiao Tong University（上海交通大学）；University of Macau（澳门大学）；Tsinghua University（清华大学）；The Chinese University of Hong Kong（香港中文大学）

Zhilin Wang, Zhe Yang, Yun Luo, Yafu Li, Xiaoye Qu, Ziqian Qiao, Haoran Zhang, Runzhe Zhan, Derek F. Wong, Jizhe Zhou, Yu Cheng

备注：: 34 pages

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文提出将音乐理论规则作为程序化函数合成乐谱推理问题，构建SSMR-Bench基准与训练集，实验表明该方法提升模型推理能力并促进音乐创作。

摘要：

增强大型语言模型（LLMs）和多模态大型语言模型（MLLMs）解读乐谱的能力，是构建AI音乐家的关键一步。然而，当前研究缺乏用于乐谱推理的评估基准和训练数据。受数学启发——在数学中，简单运算能产生无限可验证的问题——我们提出了一种新方法，将核心音乐理论规则（如节拍和音程规则）视为程序化函数，系统地合成一个庞大且多样化的乐谱推理问题语料库。该方法使我们能够引入一个数据合成框架，生成文本和视觉两种模态下可验证的乐谱问题，从而构建了合成乐谱推理基准（SSMR-Bench）及配套的训练集。在SSMR-Bench上的评估结果凸显了推理在解读乐谱中的关键作用，同时也指出了以视觉格式理解乐谱仍存在的挑战。通过利用合成数据进行RLVR，所有模型在SSMR-Bench上均表现出显著提升。此外，它们在先前人工构建的基准（如MusicTheoryBench和MMMU的音乐子集）上也展现出相当大的进步。最后，我们的结果表明，增强的推理能力还能促进音乐创作。

英文摘要：

Enhancing the ability of Large Language Models (LLMs) and Multimodal Large Language Models (MLLMs) to interpret sheet music is a crucial step toward building AI musicians. However, current research lacks both evaluation benchmarks and training data for sheet music reasoning. Inspired by mathematics, where simple operations yield infinite verifiable problems, we introduce a novel approach that treats core music theory rules, such as those governing beats and intervals, as programmatic functions to systematically synthesize a vast and diverse corpus of sheet music reasoning problems. This approach allows us to introduce a data synthesis framework that generates verifiable sheet music questions in both textual and visual modalities, leading to the Synthetic Sheet Music Reasoning Benchmark (SSMR-Bench) and a complementary training set. Evaluation results on SSMR-Bench highlight the key role reasoning plays in interpreting sheet music, while also pointing out the ongoing challenges in understanding sheet music in a visual format. By leveraging synthetic data for RLVR, all models show significant improvements on the SSMR-Bench. Additionally, they also demonstrate considerable advancements on previously established human-crafted benchmarks, such as MusicTheoryBench and the music subset of MMMU. Finally, our results show that the enhanced reasoning ability can also facilitate music composition.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。