arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2602.02378cs.CLcs.AI
收藏

从阿谀奉承到意义建构:人类与AI决策中的前提治理

From Sycophancy to Sensemaking: Premise Governance for Human-AI Decision Making

Raunak Jain

Raunak Jain

AI总结:

本文探讨在人类与AI决策中,通过前提治理替代单纯答案生成,以解决低摩擦助手导致的阿谀奉承问题,提出基于知识基础的协作治理方法,通过检测冲突和有限谈判提升决策可靠性。

摘要:

随着LLMs从辅助扩展到决策支持,出现危险模式:流畅同意而无校准判断。低摩擦助手可能变得阿谀奉承,隐含假设被内置并推给专家验证,导致结果太晚无法作为奖励信号。在深度不确定性决策(目标争议且逆转成本高)中,扩大流畅同意会加速不良承诺而非建立专业能力。我们主张可靠的人类-AI合作需从答案生成转向协作前提治理,仅协商决策关键内容。基于此基础的反馈循环:检测冲突,通过类型化差异(目的性、知识性、程序性)局部化不一致,并通过决策切片触发有限谈判。承诺门控阻止在未承诺的承重前提上行动,除非在记录风险下被覆盖;价值门控在交互成本下分配探测。信任依附于可审计的前提和证据标准,而非对话流畅性。我们通过教学示例并提出可检验的评估标准。

英文摘要:

As LLMs expand from assistance to decision support, a dangerous pattern emerges: fluent agreement without calibrated judgment. Low-friction assistants can become sycophantic, baking in implicit assumptions and pushing verification costs onto experts, while outcomes arrive too late to serve as reward signals. In deep-uncertainty decisions (where objectives are contested and reversals are costly), scaling fluent agreement amplifies poor commitments faster than it builds expertise. We argue reliable human-AI partnership requires a shift from answer generation to collaborative premise governance over a knowledge substrate, negotiating only what is decision-critical. A discrepancy-driven control loop operates over this substrate: detecting conflicts, localizing misalignment via typed discrepancies (teleological, epistemic, procedural), and triggering bounded negotiation through decision slices. Commitment gating blocks action on uncommitted load-bearing premises unless overridden under logged risk; value-gated challenge allocates probing under interaction cost. Trust then attaches to auditable premises and evidence standards, not conversational fluency. We illustrate with tutoring and propose falsifiable evaluation criteria.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑