AI心理测量学：通过心理测量效度评估大型语言模型的心理推理能力
AI Psychometrics: Evaluating the Psychological Reasoning of Large Language Models with Psychometric Validities
AI总结：
本文通过心理测量学方法评估四个主流LLM的心理推理能力和整体效度，发现高性能模型在效度上优于其 predecessors，验证了AI心理测量学的应用价值。
摘要：
大规模语言模型（LLMs）参数庞大且神经网络深度高，使其复杂度接近人类大脑，但也使其成为难以评估和解释的"黑箱"系统。AI心理测量学旨在通过心理测量方法评估和解释人工智能系统的心理特质和过程。本文研究了AI心理测量学在评估四个主流LLM（GPT-3.5、GPT-4、LLaMA-2和LLaMA-3）的心理推理能力和整体心理测量效度的应用。使用技术接受模型（TAM），我们考察了这些模型在收敛效度、区分效度、预测效度和外部效度方面的表现。研究发现，所有模型的响应通常满足所有效度标准。此外，高性能模型如GPT-4和LLaMA-3在效度上优于其 predecessors GPT-3.5和LLaMA-2。这些结果帮助确立了应用AI心理测量学评估和解释大型语言模型的效度。
英文摘要：
The immense number of parameters and deep neural networks make large language models (LLMs) rival the complexity of human brains, which also makes them opaque ``black box'' systems that are challenging to evaluate and interpret. AI Psychometrics is an emerging field that aims to tackle these challenges by applying psychometric methodologies to evaluate and interpret the psychological traits and processes of artificial intelligence (AI) systems. This paper investigates the application of AI Psychometrics to evaluate the psychological reasoning and overall psychometric validity of four prominent LLMs: GPT-3.5, GPT-4, LLaMA-2, and LLaMA-3. Using the Technology Acceptance Model (TAM), we examined convergent, discriminant, predictive, and external validity across these models. Our findings reveal that the responses from all these models generally met all validity criteria. Moreover, higher-performing models like GPT-4 and LLaMA-3 consistently demonstrated superior psychometric validity compared to their predecessors, GPT-3.5 and LLaMA-2. These results help to establish the validity of applying AI Psychometrics to evaluate and interpret large language models.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
