arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2610.19476cs.SDcs.CLcs.MMeess.AS
收藏

LALM 后训练究竟学到了什么?重新思考音频理解与推理能力

What Does LALM Post-training Learn? Rethinking Audio Understanding and Reasoning Capabilities

National Taiwan University(台湾大学);NVIDIA(英伟达);NTU AI-Core(台湾大学AI核心中心)

Hua-Hsuan Chen, Xuanjun Chen, Wei-Chung Lu, Chih-Kai Yang, Yu-Xiang Lin, Sung-Feng Huang, Haibin Wu, Hung-yi Lee

AI总结:

该研究针对 LALM 后训练的实际学习内容开展控制实验,发现其性能提升多源于对特定问题格式的适配,而非真正的音频理解与推理能力,为 LALM 后训练的优化提供了方向。

摘要:

后训练通常能提升大型音频语言模型(LALM)在音频理解与推理基准上的性能,但这些提升可能反映的是模型对特定问题格式(如多项选择题)的适配,而非音频理解与推理能力的真正进步。为探究 LALM 从后训练中实际学到的内容,我们开展了控制实验,将音频理解的提升与对特定问题格式的适配分离开来。在不同模型和后训练方法下,我们发现,即便没有提供有信息的音频,大部分已报道的提升依然存在;但当问题格式改变时,这些提升大多会消失。这些结果表明,后训练往往教会模型利用任务特定的文本线索和答案模式,而非发展出可泛化的音频理解与推理能力。

英文摘要:

Post-training often improves the performance of large audio-language models on audio understanding and reasoning benchmarks. However, these improvements may reflect adaptation to specific question formats, such as multiple-choice questions, rather than genuine advances in audio understanding and reasoning capabilities. To investigate what LALMs actually learn from post-training, we conduct controlled experiments that disentangle improvements in audio understanding from adaptation to specific question formats. Across different models and post-training methods, we find that much of the reported improvement persists even without informative audio, but largely disappears when the question format changes. These findings suggest that post-training often teaches models to exploit task-specific textual cues and answer patterns rather than developing generalizable audio understanding and reasoning capabilities.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑