道德变迁还是噪声？论AI与时间不稳定的人类反馈对齐的相关问题
Moral Change or Noise? On Problems of Aligning AI With Temporally Unstable Human Feedback
AI总结：
针对人类道德偏好随时间变化导致AI对齐困难的问题，本研究通过肾脏分配场景实验量化了偏好的响应与模型不稳定性，证实其会降低AI预测性能，指出需明确动态场景下的对齐目标。
摘要：
道德领域的对齐方法旨在引出人类利益相关者的道德偏好并将其整合到AI系统中，这一思路预设道德偏好是静态目标，但这类偏好往往会随时间演变。理想状态下，让AI与动态人类偏好正确对齐应当考虑道德推理的“合理”变化，同时忽略与注意力缺陷、认知偏差或其他任意因素相关的变化。然而，常见的AI对齐方法大多忽略偏好的时间变化，这给正确对齐带来了严峻挑战，尤其是在医疗等高风险AI应用场景中，对齐失当会损害系统可信度，造成严重的个体与社会危害。本研究探究了人们的道德偏好随时间变化的程度，以及这类变化对AI对齐的影响。我们的研究以肾脏分配领域为基础，在3到5轮实验中收集了400余名参与者对假设肾移植患者配对比较的反馈。我们发现，平均而言，参与者在不同时间面对相同场景时会有6%-20%的概率改变回答（表现为“响应不稳定性”）。此外，我们观察到部分参与者的拟合决策模型随时间发生显著偏移（即“模型不稳定性”）。简单AI模型的预测性能会随响应不稳定性和模型不稳定性的上升而下降，且预测性能会随时间推移衰减，凸显了训练过程中考虑偏好时间变化的重要性。这些发现提出了与AI对齐相关的根本性规范与技术挑战，表明当用户偏好随时间发生显著变化时，需要更好地理解对齐的对象（即对齐的目标是什么）。
英文摘要：
Alignment methods in moral domains seek to elicit moral preferences of human stakeholders and incorporate them into AI. This presupposes moral preferences as static targets, but such preferences often evolve over time. Proper alignment of AI to dynamic human preferences should ideally account for "legitimate" changes to moral reasoning, while ignoring changes related to attention deficits, cognitive biases, or other arbitrary factors. However, common AI alignment approaches largely neglect temporal changes in preferences, posing serious challenges to proper alignment, especially in high-stakes applications of AI, e.g., in healthcare domains, where misalignment can jeopardize the trustworthiness of the system and yield serious individual and societal harms. This work investigates the extent to which people's moral preferences change over time, and the impact of such changes on AI alignment. Our study is grounded in the kidney allocation domain, where we elicit responses to pairwise comparisons of hypothetical kidney transplant patients from over 400 participants across 3-5 sessions. We find that, on average, participants change their response to the same scenario presented at different times around 6-20% of the time (exhibiting "response instability"). Additionally, we observe significant shifts in several participants' retrofitted decision-making models over time (capturing "model instability"). The predictive performance of simple AI models decreases as a function of both response and model instability. Moreover, predictive performance diminishes over time, highlighting the importance of accounting for temporal changes in preferences during training. These findings raise fundamental normative and technical challenges relevant to AI alignment, highlighting the need to better understand the object of alignment (what to align to) when user preferences change significantly over time.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
