信心缺失：人机对齐中的道德不确定性
Dropouts in Confidence: Moral Uncertainty in Human-LLM Alignment
AI总结：
本研究探讨道德不确定性对人机对齐的影响，通过分析32个开源模型在电车难题中的响应，发现模型架构和训练方法主导道德不确定性，提出的推理时dropout机制可改善对齐效果。
摘要：
人类在面对道德困境时表现出显著的不确定性，但此类不确定性在机器和AI智能体中的程度仍未得到充分探索。近期研究证实了机器生成响应存在过度自信的倾向，尤其是在大型语言模型（LLMs）中。随着这些系统越来越多地嵌入伦理决策场景，理解其道德推理及构建可靠AI系统的内在不确定性至关重要。本研究探讨了不确定性如何影响经典电车难题中的道德决策，分析了32个开源模型和9个不同道德维度的响应。我们首先发现，模型间的置信度差异大于道德维度内的差异，表明道德不确定性主要由模型架构和训练方法决定。为量化不确定性，我们将二元熵测量为总熵、条件熵和互信息的线性组合。为检验其影响，我们通过推理时的“dropout”为模型引入随机性。研究结果显示，该机制增加了总熵，主要通过互信息的上升，而条件熵基本保持不变。此外，该机制显著改善了人机道德对齐，互信息与对齐分数变化存在相关性。我们的结果强调，通过刻意调节不确定性并降低LLMs在道德复杂场景中的信心，有望更好地对齐模型生成决策与人类偏好。
英文摘要：
Humans display significant uncertainty when confronted with moral dilemmas, yet the extent of such uncertainty in machines and AI agents remains underexplored. Recent studies have confirmed the overly confident tendencies of machine-generated responses, particularly in large language models (LLMs). As these systems are increasingly embedded in ethical decision-making scenarios, it is important to understand their moral reasoning and the inherent uncertainties in building reliable AI systems. This work examines how uncertainty influences moral decisions in the classical trolley problem, analyzing responses from 32 open-source models and 9 distinct moral dimensions. We first find that variance in model confidence is greater across models than within moral dimensions, suggesting that moral uncertainty is predominantly shaped by model architecture and training method. To quantify uncertainty, we measure binary entropy as a linear combination of total entropy, conditional entropy, and mutual information. To examine its effects, we introduce stochasticity into models via "dropout" at inference time. Our findings show that our mechanism increases total entropy, mainly through a rise in mutual information, while conditional entropy remains largely unchanged. Moreover, this mechanism significantly improves human-LLM moral alignment, with correlations in mutual information and alignment score shifts. Our results highlight the potential to better align model-generated decisions and human preferences by deliberately modulating uncertainty and reducing LLMs' confidence in morally complex scenarios.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
