arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2510.07293cs.SDcs.AIcs.CLeess.AS
收藏

AudioMarathon:音频大语言模型长上下文音频理解与效率的综合基准

AudioMarathon: A Comprehensive Benchmark for Long-Context Audio Understanding and Efficiency in Audio LLMs

Shanghai Jiao Tong University(上海交通大学);Shanghai AI Laboratory(上海人工智能实验室);Northeastern University(东北大学);Carnegie Mellon University(卡内基梅隆大学);University of Chinese Academy of Sciences(中国科学院大学);Tsinghua University(清华大学);Sun Yat-sen University(中山大学)
Peize He, Zichen Wen, Yubo Wang, Yuxuan Wang, Xiaoqian Liu, Jiajie Huang, Zehui Lei, Zhuangcheng Gu, Xiangqi Jin, Jiabing Yang, Kai Li, Zhifei Liu, Weijia Li, C… 展开作者

Peize He, Zichen Wen, Yubo Wang, Yuxuan Wang, Xiaoqian Liu, Jiajie Huang, Zehui Lei, Zhuangcheng Gu, Xiangqi Jin, Jiabing Yang, Kai Li, Zhifei Liu, Weijia Li, Cunxiang Wang, Conghui He, Linfeng Zhang

AI总结:

针对长音频理解与效率评估的空白,提出AudioMarathon基准,涵盖长上下文、多领域与复杂推理任务,揭示现有LALMs性能随长度下降及架构改进需求。

摘要:

处理长格式音频是大型音频语言模型(LALMs)面临的主要挑战。这些模型在注意力机制的二次方成本($O(N^2)$)以及长时程时间依赖建模方面存在困难。现有的音频基准大多基于短片段构建,未能在现实的长上下文场景中评估模型。为弥补这一空白,我们引入了AudioMarathon,一个旨在评估长格式音频理解能力和推理效率的基准。AudioMarathon提供了基于三大支柱构建的多样化任务:时长从90.0秒到300.0秒的长上下文音频输入,分别对应2,250至7,500个音频令牌的编码序列;覆盖语音、声音和音乐的完整领域;以及需要多跳推理的复杂推理任务。我们评估了最先进的LALMs,并观察到随着音频长度增加,性能明显下降。我们还研究了加速技术,并分析了令牌剪枝和KV缓存驱逐的权衡。结果表明,当前LALMs之间存在较大差距,并凸显了对更好时间推理和内存高效架构的需求。我们相信AudioMarathon将推动音频和多模态研究社区开发更先进的音频理解模型,以解决复杂的音频任务。

英文摘要:

Processing long-form audio is a major challenge for Large Audio Language models (LALMs). These models struggle with the quadratic cost of attention ($O(N^2)$) and with modeling long-range temporal dependencies. Existing audio benchmarks are built mostly from short clips and do not evaluate models in realistic long context settings. To address this gap, we introduce AudioMarathon, a benchmark designed to evaluate both understanding and inference efficiency on long-form audio. AudioMarathon provides a diverse set of tasks built upon three pillars: long-context audio inputs with durations ranging from 90.0 to 300.0 seconds, which correspond to encoded sequences of 2,250 to 7,500 audio tokens, respectively, full domain coverage across speech, sound, and music, and complex reasoning that requires multi-hop inference. We evaluate state-of-the-art LALMs and observe clear performance drops as audio length grows. We also study acceleration techniques and analyze the trade-offs of token pruning and KV cache eviction. The results show large gaps across current LALMs and highlight the need for better temporal reasoning and memory-efficient architectures. We believe AudioMarathon will drive the audio and multimodal research community to develop more advanced audio understanding models capable of solving complex audio tasks.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑