arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2603.04161cs.CL
收藏

大语言模型中社会能力的踪迹

Traces of Social Competence in Large Language Models

Leiden Institute of Advanced Computer Science（莱顿先进计算机科学研究所）；Leiden University（莱顿大学）

Tom Kouwenhoven, Michiel van der Meer, Max van Duijn

备注：
Presented at the 2026 Conference on Computational Natural Language Learning (CoNLL)
阅读 PDF
arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文研究了大语言模型在虚假信念测试中的表现，通过贝叶斯逻辑回归分析模型大小和训练方法对社会认知能力的影响，发现模型规模扩大有助于性能提升，但并非绝对，同时指出解释命题态度会改变响应模式，进一步的推理导向微调会加剧这种影响。

摘要：

虚假信念测试（FBT）一直是评估理论自我（ToM）及相关社会认知能力的主要方法。对于大语言模型（LLMs），由于数据污染、模型细节不足和控制不一致等问题，该测试的可靠性和解释潜力一直有限。我们通过在192个FBT变体（Trott等人，2023）的平衡数据集上测试17个开源模型，并使用贝叶斯逻辑回归来识别模型大小和训练后对社会认知能力的影响。我们发现模型规模扩大有助于性能提升，但并非严格正比。交叉效应显示，解释命题态度（X thinks）根本上改变了响应模式。指令微调部分缓解了这种影响，但进一步的推理导向微调会加剧这种影响。在分析OLMo 2训练过程中社会推理能力的案例研究中，我们发现这种交叉效应出现在预训练阶段，表明模型在预训练过程中获取了与心理状态词汇相关的刻板响应模式，这些模式可能超过其他情境语义。最后，向量引导使我们能够将think向量作为观察到的FBT行为的因果驱动因素。

英文摘要：

The False Belief Test (FBT) has been the main method for assessing Theory of Mind (ToM) and related socio-cognitive competencies. For Large Language Models (LLMs), the reliability and explanatory potential of this test have remained limited due to issues like data contamination, insufficient model details, and inconsistent controls. We address these issues by testing 17 open-weight models on a balanced set of 192 FBT variants (Trott et al., 2023) using Bayesian Logistic regression to identify how model size and post-training affect socio-cognitive competence. We find that scaling model size benefits performance, but not strictly. A cross-over effect reveals that explicating propositional attitudes (X thinks) fundamentally alters response patterns. Instruction tuning partially mitigates this effect, but further reasoning-oriented fine-tuning amplifies it. In a case study analysing social reasoning ability throughout OLMo 2 training, we show that this cross-over effect emerges during pre-training, suggesting that models acquire stereotypical response patterns tied to mental-state vocabulary that can outweigh other scenario semantics. Finally, vector steering allows us to isolate a think vector as the causal driver of observed FBT behaviour.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

↑