大语言模型人格测量中的持久不稳定性：规模、推理和对话历史的影响
Persistent Instability in LLM's Personality Measurements: Effects of Scale, Reasoning, and Conversation History
AI总结：
研究发现大语言模型在人格测量中存在持续不稳定性，规模扩大、推理和对话历史等干预措施未能有效提升稳定性。
摘要：
大型语言模型需要一致的行为模式以确保安全部署，但存在表明大型变异性可能导致这些模型在表现人格特征时不稳定。我们提出了PERSIST（合成文本中的人格稳定性），一个全面的评估框架，测试了25个开源模型（1B-685B参数）在200万多个响应中的表现。使用传统（BFI、SD3）和新型LLM适应的人格问卷，我们系统地变化模型大小、身份、推理模式、问题顺序或改写，以及对话历史。我们的发现挑战了基本假设：（1）仅重新排列问题可引起人格测量的大幅变化；（2）扩大规模提供有限的稳定性提升：即使400B+模型在5分量表上标准差仍大于0.3；（3）预期稳定行为的干预，如推理和包含对话历史，反而可能增加变异性；（4）详细的身分指令产生混合效果，不一致的身分显示比有帮助的助手基线显著更高的变异性；（5）LLM适应的问卷尽管生态效度提高，但稳定性与以人类为中心的版本相当。这种在尺度和缓解策略中的持续不稳定性表明，当前LLM缺乏真实行为一致性的建筑基础。对于需要可预测行为的安全关键应用，这些发现表明当前的对齐策略可能不足。
英文摘要：
Large language models require consistent behavioral patterns for safe deployment, yet there are indications of large variability that may lead to an instable expression of personality traits in these models. We present PERSIST (PERsonality Stability in Synthetic Text), a comprehensive evaluation framework testing 25 open-source models (1B-685B parameters) across 2 million+ responses. Using traditional (BFI, SD3) and novel LLM-adapted personality questionnaires, we systematically vary model size, personas, reasoning modes, question order or paraphrasing, and conversation history. Our findings challenge fundamental assumptions: (1) Question reordering alone can introduce large shifts in personality measurements; (2) Scaling provides limited stability gains: even 400B+ models exhibit standard deviations >0.3 on 5-point scales; (3) Interventions expected to stabilize behavior, such as reasoning and inclusion of conversation history, can paradoxically increase variability; (4) Detailed persona instructions produce mixed effects, with misaligned personas showing significantly higher variability than the helpful assistant baseline; (5) The LLM-adapted questionnaires, despite their improved ecological validity, exhibit instability comparable to human-centric versions. This persistent instability across scales and mitigation strategies suggests that current LLMs lack the architectural foundations for genuine behavioral consistency. For safety-critical applications requiring predictable behavior, these findings indicate that current alignment strategies may be inadequate.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
