发表机构
Huazhong University of Science and Technology（华中科技大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
本研究通过雅可比透镜和方向性干预，揭示Qwen2.5-Omni-7B中“狗”概念在生成前紧邻的后期层（L22、L24）以任务依赖方式涌现，影响音频分类与描述输出。
AI 中文摘要
多模态大语言模型能够回答音频问题，但它们如何表征听觉语义并在决策中利用这些语义仍不清楚，这限制了我们对其响应形成过程的理解。我们使用雅可比透镜（J-lens）读出和方向性干预，研究了Qwen2.5-Omni-7B中的狗叫声处理。我们将狗方向定义为由J-lens导出的、与狗相关的隐藏状态向量；增加或移除其分量会调节与狗相关的信息。我们发现，该信息可在没有狗/吠叫提示线索或动物识别要求的情况下被解码。方向性干预改变了响应倾向及部分最终答案，其效应集中在生成前紧邻的后期层状态中，涉及物种分类、发声分类和声音描述。在动物/其他分类中，狗方向相对于对照未显示出可比优势。这些结果提供了因果干预证据，表明狗方向以任务依赖的方式影响输出分数，且在生成前紧邻的L22和L24层中最为一致。
英文摘要
Multimodal large language models answer audio questions, but how they represent auditory semantics and use them in decisions remains unclear, limiting our understanding of response formation. We study dog barking in Qwen2.5-Omni-7B using Jacobian lens (J-lens) readout and directional interventions. We define the dog direction as a J-lens-derived hidden-state vector associated with dog; adding or removing its component modulates dog-related information. We find this information decodable without dog/bark prompt cues or animal-identification requirements. Directional interventions change response tendencies and some final answers, with effects concentrated in late-layer states immediately before generation across species classification, vocalization classification, and sound description. The dog direction shows no comparable advantage over controls in animal/other classification. These results provide causal-intervention evidence that the dog direction affects output scores in a task-dependent manner, most consistently at L22 and L24 immediately before generation.
