arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2602.22253cs.SD
收藏

AR&D: 一种用于音频大语言模型解释的检索与描述框架

AR&D: A Framework for Retrieving and Describing Concepts for Interpreting AudioLLMs

Australian Institute for Machine Learning, University of Adelaide(澳大利亚机器学习研究所,阿德莱德大学);Dolby Laboratories(杜比实验室);School of Computer and Mathematical Sciences, University of Adelaide, Australia(计算机与数学科学学院,阿德莱德大学,澳大利亚)

Townim Faisal Chowdhury, Ta Duc Huy, Siqi Pan, Jeremy Stoddard, Zhibin Liao

AI总结:

AR&D框架通过稀疏自编码器解构音频大语言模型的多义激活,实现对模型内部特征的可解释性增强,为高风险领域应用提供可靠部署基础。

摘要:

尽管在音频感知任务中表现出色,大型音频-语言模型(AudioLLMs)仍然难以解释。这一缺乏可解释性的主要原因是这些模型中的单个神经元经常对多个无关概念做出响应。我们介绍了首个针对AudioLLMs的机理可解释性框架,利用稀疏自编码器(SAEs)将多义激活分解为单义特征。我们的流程识别代表性音频片段,通过自动标注赋予有意义的名称,并通过人工评估和引导验证概念。实验表明,AudioLLMs编码了结构化且可解释的特征,增强了透明性和可控性。这项工作为在高风险领域可靠部署提供了基础,并使未来向更大模型、多语言音频以及更精细的语调特征扩展成为可能。项目URL:https://townim-faisal.github.io/AutoInterpret-AudioLLM/

英文摘要:

Despite strong performance in audio perception tasks, large audio-language models (AudioLLMs) remain opaque to interpretation. A major factor behind this lack of interpretability is that individual neurons in these models frequently activate in response to several unrelated concepts. We introduce the first mechanistic interpretability framework for AudioLLMs, leveraging sparse autoencoders (SAEs) to disentangle polysemantic activations into monosemantic features. Our pipeline identifies representative audio clips, assigns meaningful names via automated captioning, and validates concepts through human evaluation and steering. Experiments show that AudioLLMs encode structured and interpretable features, enhancing transparency and control. This work provides a foundation for trustworthy deployment in high-stakes domains and enables future extensions to larger models, multilingual audio, and more fine-grained paralinguistic features. Project URL: https://townim-faisal.github.io/AutoInterpret-AudioLLM/

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑