LLM是复杂的伦理困境分析器吗？
Are LLMs complicated ethical dilemma analyzers?
AI总结：
本文构建含196个真实伦理困境的基准，用复合指标比较前沿LLM、专家与非专家回答，发现LLM结构对齐更强但历史依据和细致策略仍有限。
摘要：
大型语言模型（LLM）研究中的一个开放问题是，它们能否模拟人类伦理推理，并作为可信的人类判断代理。为研究这一点，我们引入了一个包含196个真实世界伦理困境和专家意见的基准数据集，每个困境被划分为五个结构化组成部分：引言、关键因素、历史理论视角、解决策略和关键要点。我们还收集了非专家人类回答用于比较；由于其简短性，这些回答仅限于关键因素部分。我们使用基于BLEU、Damerau-Levenshtein距离、TF-IDF余弦相似度和Universal Sentence Encoder相似度的复合指标框架，评估了多个前沿LLM（GPT-4o-mini、Claude-3.5-Sonnet、Deepseek-V3、Gemini-1.5-Flash）。指标权重通过基于反转的排序对齐和成对AHP分析计算，从而能够对模型输出与专家回答进行细粒度比较。结果表明，LLM在词汇和结构对齐方面总体优于非专家人类，其中GPT-4o-mini在所有部分表现最稳定。然而，所有模型在历史依据支撑和提出细致解决策略方面都存在困难，这些能力需要语境抽象。人类回答虽然结构性较弱，但有时能达到相近的语义相似度，表明存在直觉性道德推理。这些发现凸显了LLM在伦理决策中的优势与当前局限。
英文摘要：
One open question in the study of Large Language Models (LLMs) is whether they can emulate human ethical reasoning and act as believable proxies for human judgment. To investigate this, we introduce a benchmark dataset comprising 196 real-world ethical dilemmas and expert opinions, each segmented into five structured components: Introduction, Key Factors, Historical Theoretical Perspectives, Resolution Strategies, and Key Takeaways. We also collect non-expert human responses for comparison, limited to the Key Factors section due to their brevity. We evaluate multiple frontier LLMs (GPT-4o-mini, Claude-3.5-Sonnet, Deepseek-V3, Gemini-1.5-Flash) using a composite metric framework based on BLEU, Damerau-Levenshtein distance, TF-IDF cosine similarity, and Universal Sentence Encoder similarity. Metric weights are computed through an inversion-based ranking alignment and pairwise AHP analysis, enabling fine-grained comparison of model outputs to expert responses. Our results show that LLMs generally outperform non-expert humans in lexical and structural alignment, with GPT-4o-mini performing most consistently across all sections. However, all models struggle with historical grounding and proposing nuanced resolution strategies, which require contextual abstraction. Human responses, while less structured, occasionally achieve comparable semantic similarity, suggesting intuitive moral reasoning. These findings highlight both the strengths and current limitations of LLMs in ethical decision-making.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
