道德图灵测试：评估人类与LLM在道德决策中的一致性
The Moral Turing Test: Evaluating Human-LLM Alignment in Moral Decision-Making
AI总结：
该研究通过构建道德情境回应语料库和230人用户研究，发现人类与LLM道德评估存在不一致，LLM对个人框架更敏感，且人类虽偏好LLM评估但存在系统性反AI偏见，揭示了道德决策中人类-AI感知的复杂性。
摘要：
随着大型语言模型（LLMs）日益融入社会，其与人类道德的一致性至关重要。为更好地理解这种一致性，我们构建了一个包含人类和LLM对各种道德情境生成回应的大型语料库。我们发现人类与LLM的道德评估存在不一致；尽管LLMs和人类都倾向于拒绝道德复杂的功利主义困境，但LLMs对个人框架更为敏感。随后我们进行了一项定量用户研究，涉及230名参与者（N=230），他们通过判断回应是否为AI生成来评估这些回应，并评估自身对回应的认同度。人类评估者在道德情境中更偏好LLMs的评估，尽管观察到系统性的反AI偏见：参与者较不可能认同他们认为是由机器生成的判断。基于统计和NLP的分析揭示了回应中细微的语言差异，这些差异影响了检测和认同。总体而言，我们的发现突显了在道德负荷的决策中人类-AI感知的复杂性。
英文摘要：
As large language models (LLMs) become increasingly integrated into society, their alignment with human morals is crucial. To better understand this alignment, we created a large corpus of human- and LLM-generated responses to various moral scenarios. We found a misalignment between human and LLM moral assessments; although both LLMs and humans tended to reject morally complex utilitarian dilemmas, LLMs were more sensitive to personal framing. We then conducted a quantitative user study involving 230 participants (N=230), who evaluated these responses by determining whether they were AI-generated and assessed their agreement with the responses. Human evaluators preferred LLMs' assessments in moral scenarios, though a systematic anti-AI bias was observed: participants were less likely to agree with judgments they believed to be machine-generated. Statistical and NLP-based analyses revealed subtle linguistic differences in responses, influencing detection and agreement. Overall, our findings highlight the complexities of human-AI perception in morally charged decision-making.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
