大语言模型是否理解人格？通过结构化行为推理重新思考人格保真度评估
Do LLMs Understand Personality? Rethinking Persona Fidelity Evaluation through Structured Behavioral Inference
浏览论文内容
中文总结 AI 辅助
针对现有LLM人格评估范式的局限，提出基于SFL的PRISM框架，将人格保真度评估转化为结构化逆推理任务，实验显示其评估结果更准确稳定。
中文摘要 AI 辅助
随着大语言模型（LLMs）越来越多地被用于模拟多样化的人类角色，确保人格保真度已成为一项关键要求。人格保真度被定义为智能体的行为在多大程度上一致反映目标人格的心理和风格特征。然而，现有的评估范式主要依赖于整体式LLM评判者，这类评判者容易出现“整体评估幻觉”，或者依赖于静态心理测量量表，这类量表无法捕捉动态对话中所需的依赖于上下文的保真度。为解决这些局限，我们提出PRISM（基于逆系统功能语言学建模的人格推理），这是一个基于心理语言学的框架，将人格保真度评估重新表述为结构化逆推理任务。受系统功能语言学（SFL）启发，PRISM将人格保真度分解为三个功能维度：任务框架、人际立场和语言风格。它在人格条件标签空间上估计特定维度的证据，并将这些信号聚合为可解释且可审计的评估过程。实验表明，PRISM比传统的整体式评判产生更准确、更稳定的判断，为人格保真度评估提供了更可靠的框架。
英文摘要
As large language models are increasingly deployed to simulate diverse human characters, ensuring persona fidelity, defined as the extent to which an agent's behavior consistently reflects the psychological and stylistic characteristics of a target persona, has become a critical requirement. However, existing evaluation paradigms primarily rely on either holistic LLM-based judges, which are prone to "holistic appraisal hallucination'', or static psychometric inventories, which fail to capture the context-dependent fidelity required in dynamic dialogue. To address these limitations, we propose PRISM (Persona Reasoning with Inverse SFL-based Modeling), a psycholinguistically grounded framework that reformulates persona fidelity evaluation as a structured inverse inference task. Inspired by Systemic Functional Linguistics (SFL), PRISM decomposes persona fidelity into three functional dimensions: Task Framing, Interpersonal Stance, and Linguistic Style. It estimates dimension-specific evidence over a persona-conditioned label space and aggregates these signals into an interpretable and auditable evaluation process. Experiments show that PRISM yields more accurate and stable judgements than traditional holistic judging, providing a more reliable framework for persona fidelity evaluation.
发表机构
- Singapore Management University（新加坡管理大学）
机构由 AI 辅助整理，请以论文原文为准。
