arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2108.12973cs.SDcs.MMeess.AS
收藏

Armor:人工音乐元评估的基准

Armor: A Benchmark for Meta-evaluation of Artificial Music

University of North Carolina at Chapel Hill(北卡罗来纳大学教堂山分校)

Songhe Wang, Zheng Bao, Jingtong E

摘要:

客观评价(OE)对人工音乐至关重要,但通常很难确定OE的质量。然而,主观评价(SE)仍然是可靠的和普遍的,但遭受不可避免的缺点,OE可以克服。因此,设计师需要一个元评估系统来检验设计环境的有效性。在本文中,我们介绍了Armor,一个复杂的跨领域基准数据集,用于此目的。由于OEs应该与人类的判断相关,我们提供音乐作为OEs的测试案例,并将人类的判断分数作为试金石。我们还提供了两个元评估方案和相应的测试方法来评估开放式环境的有效性。据我们所知,Armor是第一个全面而严格的框架,未来的作品可以遵循,作为例子,并改进评估计算机生成音乐和整个计算音乐领域的任务。通过分析我们数据集上不同的OE方法,我们观察到SE和OE之间仍然存在巨大的差距,这意味着硬编码算法远远不能捕捉人类对音乐的判断。

英文摘要:

Objective evaluation (OE) is essential to artificial music, but it's often very hard to determine the quality of OEs. Hitherto, subjective evaluation (SE) remains reliable and prevailing but suffers inevitable disadvantages that OEs may overcome. Therefore, a meta-evaluation system is necessary for designers to test the effectiveness of OEs. In this paper, we present Armor, a complex and cross-domain benchmark dataset that serves for this purpose. Since OEs should correlate with human judgment, we provide music as test cases for OEs and human judgment scores as touchstones. We also provide two meta-evaluation scenarios and their corresponding testing methods to assess the effectiveness of OEs. To the best of our knowledge, Armor is the first comprehensive and rigorous framework that future works could follow, take example by, and improve upon for the task of evaluating computer-generated music and the field of computational music as a whole. By analyzing different OE methods on our dataset, we observe that there is still a huge gap between SE and OE, meaning that hard-coded algorithms are far from catching human's judgment to the music.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑