ChatGPT是谁？使用 PsychoBench 对大语言模型的心理刻画进行基准测试
Who is ChatGPT? Benchmarking LLMs' Psychological Portrayal Using PsychoBench
AI总结：
本文提出 PsychoBench 心理评测框架，基于十三份临床心理量表从四类维度评估五个主流LLM，并通过越狱提示探究其安全对齐之下的内在心理特征。
摘要：
大语言模型（LLM）近来展现出卓越能力，不仅体现在自然语言处理任务中，也覆盖临床医学、法律咨询和教育等多个领域。LLM已不再只是应用，而逐渐成为能够处理各类用户请求的助手。这缩小了人类与人工智能智能体之间的差异，并引发了关于LLM是否可能表现出人格、气质和情绪的有趣问题。本文提出 PsychoBench 框架，用于评估LLM的多个心理方面。PsychoBench包含临床心理学中常用的十三种量表，并进一步将这些量表划分为四个不同类别：人格特质、人际关系、动机测试和情绪能力。我们的研究考察了五个流行模型，即 text-davinci-003、gpt-3.5-turbo、gpt-4、LLaMA-2-7b 和 LLaMA-2-13b。此外，我们采用一种越狱方法绕过安全对齐协议，以测试LLM的内在本质。我们已通过 https://github.com/CUHK-ARISE/PsychoBench 公开提供 PsychoBench。
英文摘要：
Large Language Models (LLMs) have recently showcased their remarkable capacities, not only in natural language processing tasks but also across diverse domains such as clinical medicine, legal consultation, and education. LLMs become more than mere applications, evolving into assistants capable of addressing diverse user requests. This narrows the distinction between human beings and artificial intelligence agents, raising intriguing questions regarding the potential manifestation of personalities, temperaments, and emotions within LLMs. In this paper, we propose a framework, PsychoBench, for evaluating diverse psychological aspects of LLMs. Comprising thirteen scales commonly used in clinical psychology, PsychoBench further classifies these scales into four distinct categories: personality traits, interpersonal relationships, motivational tests, and emotional abilities. Our study examines five popular models, namely text-davinci-003, gpt-3.5-turbo, gpt-4, LLaMA-2-7b, and LLaMA-2-13b. Additionally, we employ a jailbreak approach to bypass the safety alignment protocols and test the intrinsic natures of LLMs. We have made PsychoBench openly accessible via https://github.com/CUHK-ARISE/PsychoBench.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
