arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2606.26987cs.CLcs.AI
收藏

模型在哪里找到幸福?开源大语言模型中的情感向量

Where Do Models Find Happiness? Emotion Vectors in Open-Source LLMs

Sinie van der Ben, Raphaël Baur, Yannick Metz, Mennatallah El-Assady

AI总结:

研究在开源模型中复现情感向量,发现效价几何结构在层间分布存在差异,且唤醒度编码受语料影响。

摘要:

最近的研究在 Claude Sonnet 4.5 中识别出情感向量,这些内部表示编码情感概念、因果性地影响行为,并展现出镜像人类心理结构的几何结构。我们在两个开源权重模型 Apertus-8B-Instruct-2509 和 Gemma-4-E4B-it 中测试了这些发现的普适性,使用两个模型生成的语料库,提取所有层的情感对比向量。我们恢复了两个模型的效价几何结构,峰值 PC1-效价相关性分别为 $r = 0.76$ 和 $r = 0.83$,接近该 http URL 复制中报告的 $r = 0.81$。我们观察到效价表示在模型深度上的出现方式存在显著差异。在 Gemma-4-E4B-it 中,效价在早期层强烈编码,但在后期层崩溃,而 Apertus-8B-Instruct-2509 则呈现相反模式,效价表示在早期层缺失,但在中层出现。相比之下,唤醒度编码对提取语料敏感:两个模型在使用 Gemma 生成的故事时表现出更强的 PC2-唤醒度对齐($r$ 高达 $0.45$),而使用 Apertus 生成的故事时则较弱($r \leq 0.21$),这表明唤醒度相关线索在生成语料中分布不均。我们开源了实验代码和数据集,以便跨语言模型架构可重复地研究情感表示。

英文摘要:

Recent work identified emotion vectors in Claude Sonnet 4.5, which are internal representations that encode emotion concepts, causally influence behavior, and exhibit geometry mirroring human psychological structure. We test the generality of these findings in two open-weight models, Apertus-8B-Instruct-2509 and Gemma-4-E4B-it, extracting emotion contrast vectors across all layers, using two model-generated corpora. We recover valence geometry for both models, with peak PC1--valence correlations of $r = 0.76$ and $r = 0.83$, approaching the $r = 0.81$ reported for Claude.Beyond replication, we observe notable differences in how valence representations emerge across model depth. In Gemma-4-E4B-it, valence is strongly encoded in early layers but collapses towards later layers, whereas Apertus-8B-Instruct-2509 exhibits the opposite pattern, with valence representations absent in early layers, but emerging at mid depths. Arousal encoding, in contrast, is sensitive to the extraction corpus: both models show stronger PC2--arousal alignment with Gemma-generated stories ($r$ up to $0.45$) than Apertus-generated ones ($r \leq 0.21$), suggesting arousal-relevant cues are unevenly distributed across generated corpora. We open-source our experiment code and dataset for reproducible investigation of emotion representations across language model architectures.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑