arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2402.08341cs.CLcs.AI

在大语言模型中引出人格特质

Eliciting Personality Traits in Large Language Models

Holistic AI(整体智能公司);University College London(伦敦大学学院)

Airlie Hilliard, Cristian Munoz, Zekun Wu, Adriano Soares Koshiyama

AI总结:

本研究通过常见面试问题及“大五”人格特质提示引出大语言模型的人格,发现参数规模与开放性和尽责性正相关,微调会带来轻微人格调节。

摘要:

大语言模型(LLMs)在招聘场景中越来越多地被求职者和雇主使用。然而,这也带来了诸多伦理问题,尤其是与这些“黑盒”模型缺乏透明度相关的问题。尽管以往的研究试图通过调查LLMs的人格特质来提高这些模型的透明度,但许多研究为它们提供了人格测评表来完成。另一方面,本研究旨在通过检查模型基于不同输入提示的输出变化,来更好地理解这些模型。具体而言,我们使用一种新颖的引出方法,利用源自常见面试问题的提示,以及旨在引出特定“大五”人格特质的提示,来检查这些模型是否像人类一样容易受到特质激活的影响,并根据其输出中使用的语言来测量其人格。为此,我们反复提示了多个具有不同参数规模的LMs,包括 Llama-2、Falcon、Mistral、Bloom、GPT、OPT 和 XLNet(基础版和微调版),并使用在 myPersonality 数据集上训练的分类器检查了它们的人格。我们的结果显示,总体而言,所有LLMs都表现出高开放性和低外向性。然而,参数较少的LMs在人格特质上表现出相似的行为,而较新的和参数更多的LMs则表现出更广泛的人格特质,宜人性、情绪稳定性和开放性均有所增加。此外,更多的参数与开放性和尽责性呈正相关。而且,微调模型在其人格特质上表现出轻微的调节,这取决于所使用的数据集。本文还讨论了未来研究的启示和方向。

英文摘要:

Large Language Models (LLMs) are increasingly being utilized by both candidates and employers in the recruitment context. However, with this comes numerous ethical concerns, particularly related to the lack of transparency in these "black-box" models. Although previous studies have sought to increase the transparency of these models by investigating the personality traits of LLMs, many of the previous studies have provided them with personality assessments to complete. On the other hand, this study seeks to obtain a better understanding of such models by examining their output variations based on different input prompts. Specifically, we use a novel elicitation approach using prompts derived from common interview questions, as well as prompts designed to elicit particular Big Five personality traits to examine whether the models were susceptible to trait-activation like humans are, to measure their personality based on the language used in their outputs. To do so, we repeatedly prompted multiple LMs with different parameter sizes, including Llama-2, Falcon, Mistral, Bloom, GPT, OPT, and XLNet (base and fine tuned versions) and examined their personality using classifiers trained on the myPersonality dataset. Our results reveal that, generally, all LLMs demonstrate high openness and low extraversion. However, whereas LMs with fewer parameters exhibit similar behaviour in personality traits, newer and LMs with more parameters exhibit a broader range of personality traits, with increased agreeableness, emotional stability, and openness. Furthermore, a greater number of parameters is positively associated with openness and conscientiousness. Moreover, fine-tuned models exhibit minor modulations in their personality traits, contingent on the dataset. Implications and directions for future research are discussed.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑