发表机构
KAUST（阿卜杜拉国王科技大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
本研究提出一种仅用9个情绪类别名称和少量叙事段落即可找到跨视觉、音频等四种模态迁移的无标签效价轴的方法，该轴在多模态情感任务中表现优异且具有机制活性。
AI 中文摘要
现代语言模型内部存在一个单一的内部方向，用于追踪句子的正负情感倾向。本文展示了如何仅通过9个情绪类别名称加上每个情绪对应的50个短篇叙事段落（比常规监督方法少约1500个标签）来找到该效价轴（V轴），且该方向会出现在从未经过联合训练的视觉、音频和人脑编码器中。具体方法为：将9个以情绪为锚点的故事集嵌入到冻结编码器中，取9个平均嵌入的主方向。将新输入投影到该轴上，在SST-2数据集上的监督性能占比达93%（Llama-3-8B-Instruct，AUC为0.772，而监督方法为0.828）；与11811张EmoSet图像的人类效价评分的相关系数r=0.636；在ESC-50音频数据集上达到AUC 0.906（p<2.2e-15）；在123名受试者的EEG数据上达到AUC 0.720±0.055（p<3.65e-8）。该方向具有机制活性：消融该方向会使三个大语言模型的情感准确率下降5.5至37.2个百分点，而匹配的随机方向最多仅下降0.88个百分点（z>12）。一个基于文本标签训练的2参数分类器，无需目标模态标签即可迁移到图像（AUC 0.961）、音频（0.764）和脑电记录（0.828）；而通用16维子空间仅达到随机水平（0.525）。该方法仅限于连续属性——对分类概念的7项测试结果接近随机水平；且导向性具有家族特异性（Llama、Mistral适用，Qwen、Gemma不适用）。
英文摘要
Inside a modern language model sits a single internal direction that tracks how positive or negative a sentence feels. We show how to find this valence axis (V-axis) from just 9 emotion category names plus 50 short narrative paragraphs per emotion -- about 1,500 fewer labels than the usual supervised approach -- and that the same direction appears in vision, audio, and human-brain encoders never jointly trained. The recipe: embed nine emotion-anchored story sets in a frozen encoder, take the top principal direction of the nine averaged embeddings. Projecting new inputs onto it captures 93% of supervised performance on SST-2 (Llama-3-8B-Instruct, AUC 0.772 vs. 0.828), correlates with human valence ratings on 11,811 EmoSet images at r=0.636, reaches AUC 0.906 on ESC-50 audio (p<2.2e-15), and AUC 0.720+/-0.055 on EEG from 123 subjects (p<3.65e-8). The direction is mechanistically active: ablating it collapses sentiment accuracy by 5.5-37.2 pp across three LLMs vs. at most 0.88 pp for matched random directions (z>12). A 2-parameter classifier trained on text labels transfers to images (AUC 0.961), audio (0.764), and brain recordings (0.828) without target-modality labels; a generic 16-D subspace stays at chance (0.525). The recipe is bounded to continuous attributes -- seven tests on categorical concepts return near-chance -- and steering is family-specific (Llama/Mistral yes, Qwen/Gemma no).
Comments15 pages, 3 figures, 4 tables
