神经元听什么?一个通用音频模型的神经元层面解剖
What Do Neurons Listen To? A Neuron-level Dissection of a General-purpose Audio Model
AI总结:
本文通过神经元层面分析,揭示了通用音频SSL模型中类别特定神经元的产生机制及其对分类性能的影响。
摘要:
在本文中,我们从神经元层面分析了一个通用的音频自监督学习(SSL)模型的内部表示。尽管这些模型在作为特征提取器方面表现出强大的实证性能,但SSL音频模型稳健泛化背后的内部机制仍然不明确。借助机制可解释性框架,我们通过分析跨多样任务的条件激活模式,识别并检验了类别特定的神经元。我们的分析揭示了SSL模型促进了在新任务类别中广泛覆盖的类别特定神经元的出现。这些神经元在不同的语义类别和声音相似性上表现出共享的响应,例如语音属性和音乐音高。我们还确认这些神经元对分类性能有功能影响。到目前为止,这第一次对通用音频SSL模型进行了系统性的神经元层面分析,为理解其内部表示提供了新的见解。
英文摘要:
In this paper, we analyze the internal representations of a general-purpose audio self-supervised learning (SSL) model from a neuron-level perspective. Despite their strong empirical performance as feature extractors, the internal mechanisms underlying the robust generalization of SSL audio models remain unclear. Drawing on the framework of mechanistic interpretability, we identify and examine class-specific neurons by analyzing conditional activation patterns across diverse tasks. Our analysis reveals that SSL models foster the emergence of class-specific neurons that provide extensive coverage across novel task classes. These neurons exhibit shared responses across different semantic categories and acoustic similarities, such as speech attributes and musical pitch. We also confirm that these neurons have a functional impact on classification performance. To our knowledge, this is the first systematic neuron-level analysis of a general-purpose audio SSL model, providing new insights into its internal representation.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。