arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2409.09601cs.SDcs.AIcs.MMeess.AS
收藏

音乐理解基础模型综述

A Survey of Foundation Models for Music Understanding

School of Automation, Northwestern Polytechnical University(西北工业大学自动化学院);School of Physics and Information Technology, Shaanxi Normal University(陕西师范大学物理与信息技术学院)

Wenjun Li, Ying Cai, Ziyang Wu, Wenyi Zhang, Yifan Chen, Rundong Qi, Mengqi Dong, Peigen Chen, Xiao Dong, Fenghao Shi, Lei Guo, Junwei Han, Bao Ge, Tianming Liu, Lin Gan, Tuo Zhang

AI总结:

该综述聚焦AI与音乐理解交叉领域,调研分析了近期大规模音乐基础模型的音乐理解能力,讨论其局限性并提出未来方向,为相关研究者提供参考。

摘要:

音乐是日常生活中不可或缺的一部分,它满足人们的情感与娱乐需求,在个人、社会与文化层面将我们联结在一起。更好地理解音乐能够提升我们的情感体验、认知能力与文化联结感。人工智能(AI)的快速发展为音乐分析带来了新方式,其目标是复刻人类对音乐的理解并提供相关服务。传统模型聚焦于音频特征与简单任务,而近期大型语言模型(LLMs,大语言模型)和基础模型(FMs)的发展——这类模型通过整合语义信息、展现强大推理能力在各领域表现出色——能够捕捉复杂的音乐特征与模式,将音乐与语言相融合,并纳入丰富的音乐、情感与心理学知识。因此,它们有潜力从语义视角处理复杂的音乐理解任务,生成更贴近人类感知的输出。据我们所知,本研究是最早针对AI技术与音乐理解交叉领域的综述之一。我们调研、分析并测试了近期的大规模音乐基础模型的音乐理解能力,还讨论了它们的局限性并提出了可行的未来方向,为该领域的研究者提供参考见解。

英文摘要:

Music is essential in daily life, fulfilling emotional and entertainment needs, and connecting us personally, socially, and culturally. A better understanding of music can enhance our emotions, cognitive skills, and cultural connections. The rapid advancement of artificial intelligence (AI) has introduced new ways to analyze music, aiming to replicate human understanding of music and provide related services. While the traditional models focused on audio features and simple tasks, the recent development of large language models (LLMs) and foundation models (FMs), which excel in various fields by integrating semantic information and demonstrating strong reasoning abilities, could capture complex musical features and patterns, integrate music with language and incorporate rich musical, emotional and psychological knowledge. Therefore, they have the potential in handling complex music understanding tasks from a semantic perspective, producing outputs closer to human perception. This work, to our best knowledge, is one of the early reviews of the intersection of AI techniques and music understanding. We investigated, analyzed, and tested recent large-scale music foundation models in respect of their music comprehension abilities. We also discussed their limitations and proposed possible future directions, offering insights for researchers in this field.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑