超越自我报告：用于大语言模型人格评估的多观察者智能体
Beyond Self-Reports: Multi-Observer Agents for Personality Assessment in Large Language Models
AI总结：
针对大语言模型自我报告人格评估存在偏差与元知识污染的问题，提出借鉴心理学知情者报告法的多观察者框架，通过不同关系背景的观察者智能体对话评估大五人格，聚合5-7名观察者反馈可降偏差提信度，提供更可靠的情境化评估方案。
摘要：
自我报告问卷长期以来被用于评估大语言模型（LLM）的人格特质，但由于偏差和元知识污染，这类方法无法捕捉行为层面的细微差异。本文借鉴心理学中的知情者报告法，提出了一种用于LLM智能体人格特质评估的新型多观察者框架。该方法不再依赖自我评估，而是采用多个观察者智能体：每个观察者都被配置了特定的关系背景（如家庭成员、朋友或同事），先与作为被试的LLM进行对话，再从大五人格维度评估其行为。研究表明，与传统自我报告相比，这类观察者报告的评分与人类判断的契合度更高，还能揭示LLM自我评估中的系统性偏差。研究还发现，聚合5到7名观察者的反馈可降低系统性偏差，达到最优信度。研究结果凸显了关系背景在人格感知中的作用，证明多观察者范式为评估LLM人格特质提供了一种更可靠、对情境敏感的方法。
英文摘要：
Self-report questionnaires have long been used to assess LLM personality traits, yet they fail to capture behavioral nuances due to biases and meta-knowledge contamination. This paper proposes a novel multi-observer framework for personality trait assessments in LLM agents that draws on informant-report methods in psychology. Instead of relying on self-assessments, we employ multiple observer agents. Each observer is configured with a specific relational context (e.g., family member, friend, or coworker) and engages the subject LLM in dialogue before evaluating its behavior across the Big Five dimensions. We show that these observer-report ratings align more closely with human judgments than traditional self-reports and reveal systematic biases in LLM self-assessments. We also found that aggregating responses from 5 to 7 observers reduces systematic biases and achieves optimal reliability. Our results highlight the role of relationship context in perceiving personality and demonstrate that a multi-observer paradigm offers a more reliable, context-sensitive approach to evaluating LLM personality traits.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
