arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2610.15136cs.AIcs.HC
收藏

信任阻碍双向人机对齐

Trust Hinders Bidirectional Human-AI Alignment

University of Zurich(苏黎世大学);USI Lugano(卢加诺大学)

Michaela Benk, Kacper Sokol

AI总结:

本文指出信任校准框架过度限制人机辅助决策的内涵,提出AI辅助决策的三个维度,反思其对面向过程的人机耦合系统设计的启示。

摘要:

信任校准仍是AI辅助决策的主流框架,其目标是缩小用户对AI系统的感知可信度与实际可信度之间的差距,使用户对AI的依赖程度恰当。我们认为,该方法虽适用于定义明确、范围清晰的决策问题,但过度限制了借助AI支持进行决策的内涵。为克服这一局限,我们提出AI辅助决策的三个维度:其一,信任AI只是管理不确定性的多种可能策略之一;其二,当AI模型处理“大世界”决策问题(即目标与状态仅部分已知或未知的定义不明确情境)时,信任校准常被误用;其三,在此类情境中,人机对齐需处理高阶决策过程。最后,我们反思上述观察对面向过程的人机耦合系统设计的启示,这类系统需在整个决策任务中共同识别并协商不确定性。

英文摘要:

Trust calibration remains the dominant framing of AI-assisted decision making: the goal is to close the gap between perceived and actual trustworthiness of AI systems so that users rely on them appropriately. We argue that this approach, while useful for decision problems that are well-defined and clearly scoped, overly restricts what it means to make a decision with AI support. To overcome this limitation, we develop three dimensions of AI-assisted decision making. First, trusting AI is only one of several possible strategies for managing uncertainty. Second, trust calibration is often misapplied when AI models large-world decision problems, i.e., ill-defined situations where goals and states are only partially known or unknowable. Third, in such settings human-AI alignment requires addressing higher-order decision processes. We conclude by reflecting on the implications of our observations on the design of process-oriented human-AI coupled systems that jointly identify and negotiate uncertainty throughout the decision-making task.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑