从潜变量视角看大语言模型中的认知幻影
Cognitive phantoms in LLMs through the lens of latent variables
AI总结：
本研究使用两份验证性人格问卷比较人类与三个大语言模型的潜在人格结构，指出现有心理测量问卷无法有效测量 LLM 的类人构念。
摘要：
大语言模型（LLM）正越来越多地进入现实世界应用，因此有必要更好地理解其行为。它们的规模和复杂性使传统评估方法变得困难，从而促使受心理学领域启发的替代方法出现。近期对 LLM 施测心理测量问卷的研究报告称，LLM 具有类人特质，这些特质可能影响 LLM 行为。然而，这种方法存在效度问题：它预先假定这些特质存在于 LLM 中，并且可以用为人类设计的工具进行测量。典型流程很少承认 LLM 中的效度问题，只是比较和解释 LLM 的平均得分。本研究通过使用两份经过验证的人格问卷，比较人类与三个 LLM 之间人格的潜在结构来调查这一问题。研究结果表明，为人类设计的问卷不能有效地测量 LLM 中类似的构念，而且这些构念可能根本不存在于 LLM 中，这凸显了需要对 LLM 反应进行心理测量分析，以避免追逐认知幻影。
英文摘要：
Large language models (LLMs) increasingly reach real-world applications, necessitating a better understanding of their behaviour. Their size and complexity complicate traditional assessment methods, causing the emergence of alternative approaches inspired by the field of psychology. Recent studies administering psychometric questionnaires to LLMs report human-like traits in LLMs, potentially influencing LLM behaviour. However, this approach suffers from a validity problem: it presupposes that these traits exist in LLMs and that they are measurable with tools designed for humans. Typical procedures rarely acknowledge the validity problem in LLMs, comparing and interpreting average LLM scores. This study investigates this problem by comparing latent structures of personality between humans and three LLMs using two validated personality questionnaires. Findings suggest that questionnaires designed for humans do not validly measure similar constructs in LLMs, and that these constructs may not exist in LLMs at all, highlighting the need for psychometric analyses of LLM responses to avoid chasing cognitive phantoms. Keywords: large language models, psychometrics, machine behaviour, latent variable modeling, validity
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
