arXiv 2206.075502023-10-29cs.CLcs.AIcs.LG

在预训练语言模型中评估与诱导人格

Evaluating and Inducing Personality in Pre-trained Language Models

Institute for Artificial Intelligence, Peking University（北京大学人工智能研究院）；Yuanpei College, Peking University（北京大学元培学院）；National Key Laboratory of General Artificial Intelligence, BIGAI（北京通用人工智能研究院通用人工智能全国重点实验室）；Beijing Jiaotong University（北京交通大学）

Guangyuan Jiang, Manjie Xu, Song-Chun Zhu, Wenjuan Han, Chi Zhang, Yixin Zhu

备注：: Accepted at NeurIPS 2023 (Spotlight)

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文借鉴大五人格理论提出机器人格量表MPI来量化评估大语言模型的行为，并设计人格提示方法P^2以可控方式诱导LLMs表现出特定且可验证的人格，为研究类人机器行为提供了新工具和证据。

摘要：

对机器行为进行标准化和量化的评估是理解大语言模型（LLMs）的关键。在本研究中，我们从心理测量学研究中汲取灵感，利用人类人格理论作为研究机器行为的工具。人格研究起源于对人类行为的哲学探索，深入探讨个体在思维、情感和行为上的差异。为了构建和理解类人的社交机器，我们有动力提出以下问题：我们能否以有原则和定量的方式利用人类心理测量测试来评估机器行为？如果可以，我们能否在LLMs中诱导出特定的人格？为了回答这些问题，我们引入了机器人格量表（Machine Personality Inventory，MPI）工具来研究机器行为；MPI遵循标准化的人格测试，建立在大五人格因素（Big Five）理论和人格评估量表之上。通过使用MPI系统地评估LLMs，我们提供了第一份证据，证明了MPI在研究LLMs行为方面的有效性。我们进一步设计了一种人格提示（Personality Prompting，P^2）方法，以可控的方式诱导LLMs具有特定的人格，能够产生多样且可验证的行为。我们希望这项工作通过将人格作为各种下游任务的重要指标，为未来的研究提供启示，并进一步推动对同样引人入胜的类人机器行为的研究。

英文摘要：

Standardized and quantified evaluation of machine behaviors is a crux of understanding LLMs. In this study, we draw inspiration from psychometric studies by leveraging human personality theory as a tool for studying machine behaviors. Originating as a philosophical quest for human behaviors, the study of personality delves into how individuals differ in thinking, feeling, and behaving. Toward building and understanding human-like social machines, we are motivated to ask: Can we assess machine behaviors by leveraging human psychometric tests in a principled and quantitative manner? If so, can we induce a specific personality in LLMs? To answer these questions, we introduce the Machine Personality Inventory (MPI) tool for studying machine behaviors; MPI follows standardized personality tests, built upon the Big Five Personality Factors (Big Five) theory and personality assessment inventories. By systematically evaluating LLMs with MPI, we provide the first piece of evidence demonstrating the efficacy of MPI in studying LLMs behaviors. We further devise a Personality Prompting (P^2) method to induce LLMs with specific personalities in a controllable way, capable of producing diverse and verifiable behaviors. We hope this work sheds light on future studies by adopting personality as the essential indicator for various downstream tasks, and could further motivate research into equally intriguing human-like machine behaviors.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。