arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2601.04876cs.SD
收藏

ChronosAudio: 用于评估音频大语言模型的综合长音频基准

ChronosAudio: A Comprehensive Long-Audio Benchmark for Evaluating Audio-Large Language Models

Nanyang Technological University(南洋理工大学);Independent Researcher(独立研究者);United Arab Emirates University(阿联酋大学);North China Electric Power University(华北电力大学);Southwest Jiaotong University(西南交通大学);Squirrel AI Learning(Squirrel AI学习)

Kaiwen Luo, Liang Lin, Yibo Zhang, Moayad Aloqaily, Jialiang Tao, Dexian Wang, Zhenhong Zhou, Junwei Zhang, Kun Wang, Li Sun, Qingsong Wen

AI总结:

提出首个针对音频大语言模型长音频理解的多任务基准ChronosAudio,包含6大任务类别和36000个测试实例,实验发现模型存在长上下文崩溃、注意力稀释等问题,现有缓解策略仅恢复50%性能。

摘要:

尽管音频大语言模型(ALLMs)取得了显著进展,但其长音频理解能力仍未得到探索。针对通用音频任务,已有大量基准被提出,但它们主要关注短视频片段,缺乏评估ALLMs在长时间跨度上的共识。本文提出ChronosAudio,这是首个针对ALLMs长音频理解定制的多任务基准。它包含六大任务类别,共36000个测试实例,总时长超过200小时,并按短、中、长三类进行分层,以全面评估长度泛化能力。使用ChronosAudio对16个最先进模型进行的广泛实验得出了三个关键发现:1. 急剧的长上下文崩溃:ALLMs表现出严重的性能维持能力不足,从短上下文过渡到长上下文时,特定任务的性能下降超过90%。2. 结构性注意力稀释:性能下降源于维持时间局部性的根本失败;注意力机制在后续序列中遭受显著扩散。3. 缓解措施的效果上限:当前策略仅能恢复50%的性能。这些发现揭示了长音频中的重大挑战,强调了迫切需要实现稳健的文档级音频推理的方法。

英文摘要:

Although Audio Large Language Models (ALLMs) have witnessed substantial advancements, their long audio understanding capabilities remain unexplored. A plethora of benchmarks have been proposed for general audio tasks, they predominantly focus on short-form clips, leaving without a consensus on evaluating ALLMs over extended durations. This paper proposes ChronosAudio, the first multi-task benchmark tailored for long-audio understanding in ALLMs. It encompasses six major task categories and comprises 36,000 test instances totaling over 200 hours audio, stratified into short, middle, and long-form categories to comprehensively evaluate length generalization. Extensive experiments on 16 state-of-the-art models using ChronosAudio yield three critical findings: 1.Precipitous Long-Context Collapse: ALLMs exhibit a severe inability to sustain performance, with the transition from short to long contexts triggering a staggering performance degradation of over 90% in specific tasks. 2.Structural Attention Dilution: Performance degradation stems from a fundamental failure in maintaining temporal locality; attention mechanisms suffer from significant diffusion in later sequences. 3.Restorative Ceiling of Mitigation: Current strategies only offer 50% recovery. These findings reveal significant challenges in long-audio, underscoring the urgent need for approaches to achieve robust, document-level audio reasoning.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑