LLMs 的道德判断与推理能力是否随语言变化？基于多语言定义问题测验的研究
Do Moral Judgment and Reasoning Capability of LLMs Change with Language? A Study using the Multilingual Defining Issues Test
AI总结：
本研究使用多语言定义问题测验，发现LLMs在印地语和斯瓦希里语中的道德推理能力显著低于其他语言，且道德判断随语言变化。
摘要：
本文通过定义问题测验（Defining Issues Test）探究了大型语言模型（LLMs）在不同语言中展现的道德判断和道德推理能力。众所周知，道德判断取决于提问所用的语言。我们将先前的工作从英语扩展到5种新语言（中文、印地语、俄语、西班牙语和斯瓦希里语），并测试了三个LLMs——ChatGPT、GPT-4和Llama2Chat-70B——这些模型具有显著的多语言文本处理和生成能力。我们的研究表明，所有模型在印地语和斯瓦希里语中的道德推理能力（以习俗后得分衡量）明显低于西班牙语、俄语、中文和英语，而后者四种语言的表现没有明显趋势。道德判断也随语言不同而有显著差异。
英文摘要：
This paper explores the moral judgment and moral reasoning abilities exhibited by Large Language Models (LLMs) across languages through the Defining Issues Test. It is a well known fact that moral judgment depends on the language in which the question is asked. We extend the work of beyond English, to 5 new languages (Chinese, Hindi, Russian, Spanish and Swahili), and probe three LLMs -- ChatGPT, GPT-4 and Llama2Chat-70B -- that shows substantial multilingual text processing and generation abilities. Our study shows that the moral reasoning ability for all models, as indicated by the post-conventional score, is substantially inferior for Hindi and Swahili, compared to Spanish, Russian, Chinese and English, while there is no clear trend for the performance of the latter four languages. The moral judgments too vary considerably by the language.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
