TTM-Bench：一种文本到音乐系统性能基准测试框架

中文摘要 AI 辅助

文本到音乐（TTM）系统越来越多地被用于从自然语言描述生成音乐音频。因此，稳健的评估至关重要，然而可靠的性能比较仍然具有挑战性。这一困难源于系统架构、所支持的条件信息和访问模式的差异，以及无法统一应用于所有系统的异构且碎片化的度量标准。为应对这些挑战，我们引入了TTM-Bench，这是一个定义通用协议的框架，用于对当代TTM系统进行系统性、可复现的性能基准测试。它从两个维度评估性能：音乐内容对齐度，通过可解释的语义、流派和音乐描述符一致性得分（针对共同音乐规范）进行量化，并以综合得分汇总；以及计算效率，以生成延迟和实时因子为特征，同时考虑本地模型的资源使用和托管服务的成本。我们通过一项初步的比较案例研究展示了该框架，说明了这些维度所捕获的互补证据。结果表明，较高的音乐内容对齐度并不系统地与较低的计算需求相吻合，这凸显了通过不同且可解释的度量标准而非简化的总体指标来评估TTM性能的重要性。

英文摘要

Text-to-music (TTM) systems are increasingly used to generate musical audio from natural-language descriptions. Robust evaluation is therefore essential, yet reliable performance comparison remains challenging. This difficulty stems from differences in system architecture, supported conditioning information, and access mode, as well as heterogeneous and fragmented metrics that cannot be applied uniformly across systems. To address these challenges, we introduce TTM-Bench, a framework that defines a common protocol for systematic, reproducible performance benchmarking of contemporary TTM systems. It evaluates performance along two dimensions: musical-content alignment, quantified by interpretable semantic, genre, and musical-descriptor agreement scores against a common musical specification and summarized by an aggregate score; and computational efficiency, characterized by generation latency and real-time factor, alongside resource use for local models and cost for hosted services. We demonstrate the framework through a preliminary comparative case study, illustrating the complementary evidence captured by these dimensions. The results show that higher musical-content alignment does not systematically coincide with lower computational demands, highlighting the importance of assessing TTM performance through distinct, interpretable measures rather than a reductive overall indicator.

发表机构

• Institute of Information Systems and Networking (ISIN), SUPSI（信息系统与网络研究所（ISIN），瑞士南部应用科学与艺术大学）
• t2b AG（t2b股份公司）

机构由 AI 辅助整理，请以论文原文为准。