超越BFI：用于增强评估LLM人格特质可靠性与有效性的CSI
Beyond BFI: The CSI for Enhanced Reliability and Validity in Evaluating LLM Personality Traits
AI总结：
针对现有LLM人格特质评估工具BFI的可靠性与有效性局限，提出适配LLM的CSI，经实验验证其可靠性更高、与模型真实输出相关性超0.85，能有效评估LLM人格特质。
摘要：
随着大语言模型（LLM）日益作为类人助手发挥作用并展现类人人格特质，理解其行为特征对负责任的AI开发至关重要。然而，现有评估工作常采用人类心理评估工具如大五人格量表（Big Five Inventory，BFI），存在两大显著局限：其一，这些方法通常缺乏可靠性，微小的提示变化即可导致测试结果不一致；其二，这些工具的理论基础源于人类研究，与LLM的计算本质不匹配，从而限制了其预测模型真实世界行为的有效性。为解决这些局限，我们提出核心情感量表（Core Sentiment Inventory，CSI），这是一种全新的人格特质评估工具，从零开始设计，专门适配LLM的独特特征。CSI支持中英文，可隐式评估模型的人格特质，提供LLM的深刻心理画像。大量实验表明：（1）CSI能有效捕捉细微的行为模式，揭示LLM在不同语言和情境下的显著行为差异；（2）与现有评估工具相比，CSI显著提升了可靠性，产生更一致、更稳健的结果；（3）CSI得分与LLM真实世界输出的相关性超过0.85，证明其在预测LLM行为方面具有强有效性。
英文摘要：
As large language models (LLMs) increasingly function as human-like assistants exhibiting human-like personality traits, understanding their behavioral characteristics becomes essential for responsible AI development. However, existing evaluation efforts, which often adapt human psychological assessments such as the Big Five Inventory (BFI), face two significant limitations. First, these approaches often lack reliability, as minor prompt variations can lead to inconsistent test results. Second, the theoretical foundations of these tools, rooted in human studies, are misaligned with the computational nature of LLMs, thereby limiting their validity in predicting real-world model behavior. To address these limitations, we introduce the Core Sentiment Inventory (CSI), a novel personality trait evaluation instrument designed from the ground up and specifically tailored to the unique characteristics of LLMs. CSI covers both English and Chinese, that implicitly evaluates models' personality traits, providing insightful psychological portraits of LLMs. Extensive experiments demonstrate that: (1) CSI effectively captures nuanced behavioral patterns, revealing significant behavioral variations in LLMs across different languages and contexts; (2) Compared to current evaluation tools, CSI significantly improves reliability, yielding more consistent and robust results; and (3) The correlation between CSI scores and LLMs' real-world outputs exceeds 0.85, demonstrating its strong validity in predicting LLM behavior.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
