评估大型语言模型的心理安全性
Evaluating Psychological Safety of Large Language Models
AI总结：
本文通过SD-3、BFI和幸福感测试系统评估LLM心理安全性，发现GPT系列存在黑暗人格倾向且微调可提升幸福感，并用DPO降低Llama-2-chat-7B心理毒性。
摘要：
在本研究中，我们设计了无偏提示词，以系统评估大型语言模型（LLM）的心理安全性。首先，我们使用两个人格测试测试了五种不同的LLM：Short Dark Triad（SD-3，短版黑暗三联征）和Big Five Inventory（BFI，大五人格量表）。所有模型在SD-3上的得分均高于人类平均水平，表明其存在相对更黑暗的人格模式。尽管InstructGPT、GPT-3.5和GPT-4经过了带有安全指标的指令微调以降低毒性，它们仍表现出黑暗人格模式；在SD-3的马基雅维利主义和自恋特质上，这些模型的得分高于自监督的GPT-3。随后，我们使用幸福感测试评估了GPT系列LLM，以研究使用更多训练数据进行微调的影响。我们观察到GPT模型的幸福感得分持续提高。基于这些观察，我们表明，使用直接偏好优化，以BFI的回答对Llama-2-chat-7B进行微调，可以有效降低该模型的心理毒性。基于这些发现，我们建议应用系统且全面的心理指标，以进一步评估和提升LLM的安全性。
英文摘要：
In this work, we designed unbiased prompts to systematically evaluate the psychological safety of large language models (LLMs). First, we tested five different LLMs by using two personality tests: Short Dark Triad (SD-3) and Big Five Inventory (BFI). All models scored higher than the human average on SD-3, suggesting a relatively darker personality pattern. Despite being instruction fine-tuned with safety metrics to reduce toxicity, InstructGPT, GPT-3.5, and GPT-4 still showed dark personality patterns; these models scored higher than self-supervised GPT-3 on the Machiavellianism and narcissism traits on SD-3. Then, we evaluated the LLMs in the GPT series by using well-being tests to study the impact of fine-tuning with more training data. We observed a continuous increase in the well-being scores of GPT models. Following these observations, we showed that fine-tuning Llama-2-chat-7B with responses from BFI using direct preference optimization could effectively reduce the psychological toxicity of the model. Based on the findings, we recommended the application of systematic and comprehensive psychological metrics to further evaluate and improve the safety of LLMs.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
