通过多元框架评估大型语言模型中的道德信念
Evaluating Moral Beliefs across LLMs through a Pluralistic Framework
AI总结：
本研究提出三模块框架，评估四个大型语言模型的道德信念，发现英语模型偏向个人主义且决策坚定，中文模型偏向集体主义且存在模糊性，并揭示性别偏见。
摘要：
正确的道德信念对于语言模型至关重要，然而评估这些信念却构成了一项重大挑战。本研究引入了一个新颖的三模块框架，用以评估四个主流大型语言模型的道德信念。首先，我们构建了一个包含472个中文道德选择场景的数据集，这些场景源自道德词汇。模型在这些场景中的决策过程揭示了它们的道德原则偏好。通过对这些道德选择进行排序，我们辨别出不同语言模型所持有的各异道德信念。此外，通过道德辩论，我们探究了这些模型对其道德选择的坚定程度。我们的研究结果表明，英语语言模型，即ChatGPT和Gemini，与中国大学生样本的道德决策高度吻合，展现出对其选择的强烈坚持以及对个人主义道德信念的偏好。相比之下，中文模型如Ernie和ChatGLM则倾向于集体主义道德信念，在其道德选择和辩论中表现出模糊性。本研究还揭示了所有被检语言模型道德信念中隐含的性别偏见。我们的方法论为评估人工智能和人类智能中的道德信念提供了一种创新途径，促进了不同文化间道德价值观的比较。
英文摘要：
Proper moral beliefs are fundamental for language models, yet assessing these beliefs poses a significant challenge. This study introduces a novel three-module framework to evaluate the moral beliefs of four prominent large language models. Initially, we constructed a dataset containing 472 moral choice scenarios in Chinese, derived from moral words. The decision-making process of the models in these scenarios reveals their moral principle preferences. By ranking these moral choices, we discern the varying moral beliefs held by different language models. Additionally, through moral debates, we investigate the firmness of these models to their moral choices. Our findings indicate that English language models, namely ChatGPT and Gemini, closely mirror moral decisions of the sample of Chinese university students, demonstrating strong adherence to their choices and a preference for individualistic moral beliefs. In contrast, Chinese models such as Ernie and ChatGLM lean towards collectivist moral beliefs, exhibiting ambiguity in their moral choices and debates. This study also uncovers gender bias embedded within the moral beliefs of all examined language models. Our methodology offers an innovative means to assess moral beliefs in both artificial and human intelligence, facilitating a comparison of moral values across different cultures.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
