评估社会对齐：人格提示的大型语言模型是否表现得像人类？
Assessing Social Alignment: Do Personality-Prompted Large Language Models Behave Like Humans?
AI总结：
本研究通过米尔格拉姆实验和最后通牒博弈，评估人格提示对大型语言模型行为的影响，发现其行为与所赋予人格不一致且难以精细控制，挑战了社区对人格提示的乐观看法。
摘要：
语言建模领域的持续革命催生了各种新颖的应用，其中一些依赖于大型语言模型（LLMs）新兴的社交能力。许多人已经在人生的关键时刻向这些新的网络朋友寻求建议，并将最深的秘密托付给他们，这意味着准确塑造LLM的人格至关重要。为此，最先进的方法利用大量多样的训练数据，并提示模型采用特定的人格。我们提出两个问题：（i）人格提示的模型在被赋予某种人格时，其行为（即在面对社交情境时做出的决策）是否与所赋予的人格一致；（ii）其行为能否被精细控制。我们使用经典心理学实验——米尔格拉姆实验和最后通牒博弈——作为社交互动测试平台，并对来自4个不同供应商的开源和闭源LLM应用人格提示。我们的实验揭示了基于提示的模型行为调节的失败模式，这些模式在所有测试模型中普遍存在，并在提示扰动下持续存在。这些发现挑战了社区中普遍对人格提示持乐观态度的看法。
英文摘要：
The ongoing revolution in language modeling has led to various novel applications, some of which rely on the emerging social abilities of large language models (LLMs). Already, many turn to the new cyber friends for advice during the pivotal moments of their lives and trust them with the deepest secrets, implying that accurate shaping of the LLM's personality is paramount. To this end, state-of-the-art approaches exploit a vast variety of training data, and prompt the model to adopt a particular personality. We ask (i) if personality-prompted models behave (i.e., make decisions when presented with a social situation) in line with the ascribed personality (ii) if their behavior can be finely controlled. We use classic psychological experiments, the Milgram experiment and the Ultimatum Game, as social interaction testbeds and apply personality prompting to open- and closed-source LLMs from 4 different vendors. Our experiments reveal failure modes of the prompt-based modulation of the models' behavior that are shared across all models tested and persist under prompt perturbations. These findings challenge the optimistic sentiment toward personality prompting generally held in the community.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
