心念识读还是误读？LLMs在Big Five人格测试中的表现
Mind Reading or Misreading? LLMs on the Big Five Personality Test
AI总结：
研究评估了LLMs在Big Five人格测试中的表现，发现增强提示虽能提升性能但引入系统偏见，不同特质检测难度差异显著，需协调提示设计与评估指标以获得可解释结果。
摘要：
我们评估了大型语言模型（LLMs）在从文本自动预测人格特征方面的性能，基于二元五因素模型（BIG5）。五种模型——包括GPT-4和轻量级开源替代品——在三个异质数据集（Essays、MyPersonality、Pandora）和两种提示策略（最小化与增强的言语和心理线索）下进行测试。增强提示减少了无效输出并改善了类别平衡，但同时也引入了系统性偏见，倾向于预测特质存在。性能差异显著：开放性和宜人性相对容易检测，而外向性和神经质则仍具挑战性。尽管开源模型有时能接近GPT-4和先前基准，但没有配置能在零样本二元设置中持续可靠地进行预测。此外，准确率和宏F1等聚合指标掩盖了显著的不对称性，而每类召回率提供了更清晰的诊断价值。这些发现表明，当前的现成LLMs尚不适用于自动人格预测（APPT），并且需要精心协调提示设计、特质框架和评估指标，以获得可解释的结果。
英文摘要：
We evaluate large language models (LLMs) for automatic personality prediction from text under the binary Five Factor Model (BIG5). Five models -- including GPT-4 and lightweight open-source alternatives -- are tested across three heterogeneous datasets (Essays, MyPersonality, Pandora) and two prompting strategies (minimal vs. enriched with linguistic and psychological cues). Enriched prompts reduce invalid outputs and improve class balance, but also introduce a systematic bias toward predicting trait presence. Performance varies substantially: Openness and Agreeableness are relatively easier to detect, while Extraversion and Neuroticism remain challenging. Although open-source models sometimes approach GPT-4 and prior benchmarks, no configuration yields consistently reliable predictions in zero-shot binary settings. Moreover, aggregate metrics such as accuracy and macro-F1 mask significant asymmetries, with per-class recall offering clearer diagnostic value. These findings show that current out-of-the-box LLMs are not yet suitable for APPT, and that careful coordination of prompt design, trait framing, and evaluation metrics is essential for interpretable results.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
