再探大语言模型中心理量表的信度
Revisiting the Reliability of Psychological Scales on Large Language Models
AI总结：
该研究从心理学视角探究心理量表用于大语言模型的信度，通过多模型多设置实验验证其人格作答一致性，并探索了LLMs通过提示模拟不同人格的潜力。
摘要：
近期研究聚焦于从心理学视角考察大语言模型（LLMs）的特征，明确了理解其行为特征的必要性。在此背景下，对LLMs施测人格测试已成为一个值得关注的领域。然而，最初为人类设计的心理量表是否适用于LLMs，仍是一个存在争议的问题。本研究旨在确定人格测评应用于LLMs的信度，明确探究LLMs是否表现出一致的人格特质。通过对每个模型（包括GPT-3.5、GPT-4、Gemini-Pro和LLaMA-3.1）的2500种设置进行分析，结果显示各类LLMs在大五人格量表（Big Five Inventory）的作答上具有一致性，表明信度水平令人满意。此外，本研究还探究了GPT-3.5模拟多种人格、代表不同群体的潜力——在社会科学领域，人们越来越追求这种能力，即使用LLMs替代人类参与者以降低成本。研究结果表明，通过特定的提示指令，LLMs有潜力呈现不同的人格。
英文摘要：
Recent research has focused on examining Large Language Models' (LLMs) characteristics from a psychological standpoint, acknowledging the necessity of understanding their behavioral characteristics. The administration of personality tests to LLMs has emerged as a noteworthy area in this context. However, the suitability of employing psychological scales, initially devised for humans, on LLMs is a matter of ongoing debate. Our study aims to determine the reliability of applying personality assessments to LLMs, explicitly investigating whether LLMs demonstrate consistent personality traits. Analysis of 2,500 settings per model, including GPT-3.5, GPT-4, Gemini-Pro, and LLaMA-3.1, reveals that various LLMs show consistency in responses to the Big Five Inventory, indicating a satisfactory level of reliability. Furthermore, our research explores the potential of GPT-3.5 to emulate diverse personalities and represent various groups-a capability increasingly sought after in social sciences for substituting human participants with LLMs to reduce costs. Our findings reveal that LLMs have the potential to represent different personalities with specific prompt instructions.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
