使用心理测量量表评估与操控预训练语言模型中的潜变量构念
Assessment and manipulation of latent constructs in pre-trained language models using psychometric scales
AI总结：
本研究针对缺乏合适方法评估非对话类预训练语言模型潜在心理构念的问题，提出将标准心理问卷重构为自然语言推理提示的方法并配套代码库，在88个公开模型中验证了类人心理健康构念的存在，助力开发更可解释、可控、可信的语言模型。
摘要：
近期在大语言模型中发现了类人的人格特质，由此引出一项假设：这类模型的（已知及尚未发现的）偏差与人类的潜在心理构念相符。尽管大型对话模型可能会被诱导回答心理测量问卷，但数千个针对其他任务训练的更简单transformer模型的潜在心理构念仍无法评估，因为目前缺乏合适的心理测量方法。本文中，我们展示了如何将标准心理问卷重构为自然语言推理提示，并提供了一个代码库以支持对任意模型进行心理测量评估。我们通过88个公开可用模型的样本证明，模型中存在类人的心理健康相关构念（包括焦虑、抑郁以及心理一致感），这些构念符合人类心理学的标准理论，且表现出相似的相关性与缓解策略。借助心理学工具来解释和校正语言模型的性能，能够推动更具可解释性、可控性和可信度的模型的发展。
英文摘要：
Human-like personality traits have recently been discovered in large language models, raising the hypothesis that their (known and as yet undiscovered) biases conform with human latent psychological constructs. While large conversational models may be tricked into answering psychometric questionnaires, the latent psychological constructs of thousands of simpler transformers, trained for other tasks, cannot be assessed because appropriate psychometric methods are currently lacking. Here, we show how standard psychological questionnaires can be reformulated into natural language inference prompts, and we provide a code library to support the psychometric assessment of arbitrary models. We demonstrate, using a sample of 88 publicly available models, the existence of human-like mental health-related constructs (including anxiety, depression, and Sense of Coherence) which conform with standard theories in human psychology and show similar correlations and mitigation strategies. The ability to interpret and rectify the performance of language models by using psychological tools can boost the development of more explainable, controllable, and trustworthy models.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
