arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2602.05027cs.SDcs.AI
收藏

AudioSAE:利用稀疏自编码器理解音频处理模型

AudioSAE: Towards Understanding of Audio-Processing Models with Sparse AutoEncoders

Huawei Noah’s Ark Lab(华为诺亚实验室)

Georgii Aparin, Tasnima Sadekova, Alexey Rukhovich, Assel Yermekova, Laida Kushnareva, Vadim Popov, Kristian Kuznetsov, Irina Piontkovskaya

AI总结:

本文在Whisper和HuBERT的编码器层训练稀疏自编码器(SAE),评估其稳定性和可解释性,并展示其在特征解耦、概念擦除、语音检测优化及与人类脑电活动对齐方面的实用价值。

摘要:

稀疏自编码器(SAE)是解释神经表征的强大工具,但它们在音频领域的应用尚未充分探索。我们在Whisper和HuBERT的所有编码器层训练SAE,对其稳定性、可解释性进行了广泛评估,并展示了其实用性。超过50%的特征在随机种子间保持一致,且重建质量得以保持。SAE特征捕获了通用声学和语义信息以及特定事件,包括环境噪声和副语言声音(如笑声、低语),并有效解耦它们,仅需移除19-27%的特征即可擦除一个概念。特征引导将Whisper的虚假语音检测降低了70%,且词错误率(WER)增加可忽略不计,展示了实际应用价值。最后,我们发现SAE特征与语音感知过程中的人类脑电活动相关,表明其与人类神经处理的对齐。代码和检查点可在https://github.com/audiosae/audiosae_demo获取。

英文摘要:

Sparse Autoencoders (SAEs) are powerful tools for interpreting neural representations, yet their use in audio remains underexplored. We train SAEs across all encoder layers of Whisper and HuBERT, provide an extensive evaluation of their stability, interpretability, and show their practical utility. Over 50% of the features remain consistent across random seeds, and reconstruction quality is preserved. SAE features capture general acoustic and semantic information as well as specific events, including environmental noises and paralinguistic sounds (e.g. laughter, whispering) and disentangle them effectively, requiring removal of only 19-27% of features to erase a concept. Feature steering reduces Whisper's false speech detections by 70% with negligible WER increase, demonstrating real-world applicability. Finally, we find SAE features correlated with human EEG activity during speech perception, indicating alignment with human neural processing. The code and checkpoints are available at https://github.com/audiosae/audiosae_demo.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑