大型语言模型是否发展出了个性？：自我评估测试在测量LLMs个性中的适用性
Have Large Language Models Developed a Personality?: Applicability of Self-Assessment Tests in Measuring Personality in LLMs
AI总结：
本文研究了规模1.5B至30B的五个LLMs的个性测量问题，提出选项顺序对称性作为自我评估测试可靠性的必要条件，发现LLMs存在固有偏差导致测试不可靠，呼吁开发机器个性测量工具。
摘要：
大型语言模型（LLMs）是否发展出了个性？简短的回答是响亮的“我们不知道！”。在本文中，我们表明我们目前还没有合适的工具来测量语言模型的个性。个性是影响行为的重要特征。随着LLMs在各种任务中模拟出类人智能和性能，一个自然的问题是这些模型是否发展出了个性。先前的工作通过自我评估人格测试来评估机器个性，这些测试是为评估人类个性而创建的一组多项选择题。这里的一个基本假设是，人类个性测试能够准确测量机器的个性。在本文中，我们研究了规模从1.5B到30B不等的五个LLMs中个性的出现。我们提出选项顺序对称性属性作为这些自我评估测试可靠性的必要条件。在此条件下，自我评估问题的答案与选项呈现的顺序无关。我们发现，许多LLMs的个性测试响应并不保持选项顺序对称性。我们深入研究了那些保持选项顺序对称性的LLMs测试响应，发现在这些情况下，LLMs没有考虑正在测试的情境陈述，并且无论测试的情境如何，都产生完全相同的答案。我们还识别出这些LLMs中存在固有偏差，这是上述现象的根本原因，并使得自我评估测试不可靠。这些观察表明，自我评估测试不是测量LLMs个性的正确工具。通过本文，我们希望引起对当前文献在测量LLMs个性方面缺陷的关注，并呼吁开发用于机器个性测量的工具。
英文摘要：
Have Large Language Models (LLMs) developed a personality? The short answer is a resounding "We Don't Know!". In this paper, we show that we do not yet have the right tools to measure personality in language models. Personality is an important characteristic that influences behavior. As LLMs emulate human-like intelligence and performance in various tasks, a natural question to ask is whether these models have developed a personality. Previous works have evaluated machine personality through self-assessment personality tests, which are a set of multiple-choice questions created to evaluate personality in humans. A fundamental assumption here is that human personality tests can accurately measure personality in machines. In this paper, we investigate the emergence of personality in five LLMs of different sizes ranging from 1.5B to 30B. We propose the Option-Order Symmetry property as a necessary condition for the reliability of these self-assessment tests. Under this condition, the answer to self-assessment questions is invariant to the order in which the options are presented. We find that many LLMs personality test responses do not preserve option-order symmetry. We take a deeper look at LLMs test responses where option-order symmetry is preserved to find that in these cases, LLMs do not take into account the situational statement being tested and produce the exact same answer irrespective of the situation being tested. We also identify the existence of inherent biases in these LLMs which is the root cause of the aforementioned phenomenon and makes self-assessment tests unreliable. These observations indicate that self-assessment tests are not the correct tools to measure personality in LLMs. Through this paper, we hope to draw attention to the shortcomings of current literature in measuring personality in LLMs and call for developing tools for machine personality measurement.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
