arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2610.17757cs.AIcs.SE
收藏

指令是证据而非目标:衡量智能体的结果对齐性

Instructions Are Evidence, Not Objectives: Measuring Outcome Alignment of Agents

College of AI, Tsinghua University(清华大学人工智能学院);Institute for AI Industry Research (AIR), Tsinghua University(清华大学人工智能产业研究院);Institute for AI, Tsinghua University(清华大学人工智能研究院)

Kaiming Liu, Yuxuan Liu, Jinrui Ju, Minyuan Ruan, Xuanyu Lei, Peng Li, Yang Liu

AI总结:

本研究针对大语言模型智能体的结果对齐性,构建基准TiGanEval评估8个前沿模型,发现其普遍缺失该能力,指出需将其作为下一代模型的开发方向。

摘要:

大语言模型智能体可遵循复杂人类指令解决现实任务,但实践中初始指令往往不完整。我们对来自SWE-chat的100条用户-智能体轨迹分析发现,56%的初始指令遗漏了后续被证实必要的需求,52%的有用后续需求可从项目中推断得出。用户基于自身对任务的理解陈述需求,这些「陈述需求」常与完成任务所要求的「真实需求」存在差异。因此智能体应将指令视为证据而非目标,从指令和环境中推断真实需求,即便用户尚未意识到所有需求也需满足它们,我们将此能力称为「结果对齐性」。本研究构建了软件工程与研究任务的基准TiGanEval,其中每项任务的用户均确实缺少部分需求,每项需求均可从环境的代码与运行时行为中被发现,且智能体推断出的需求与用户在观察交付成果后添加的需求相互分离,这些特性共同使结果对齐性具备可衡量性。对包括GPT-5.6-Sol在内的8个前沿模型评估后发现,结果对齐性普遍缺失:当从完整规格转向部分指令时,模型的成功率下降53至67个百分点,且四次提交的结果反馈无法缩小该差距;此外,计划模式工具无法弥补模型自身缺失的这一能力,这使得结果对齐性成为下一代模型应着力开发的能力之一。

英文摘要:

Large language model agents can follow complex human instructions to solve real-world tasks. In practice, however, initial instructions are often incomplete. In our analysis of 100 user-agent trajectories from SWE-chat, 56\% of initial instructions omit requirements that later emerge as necessary, and 52\% of the useful follow-up requirements could have been inferred from the project. A user states requirements from their own understanding of the task, and these \textbf{stated requirements} often differ from the \textbf{true requirements} that completing the task imposes. An agent should therefore treat the instruction as evidence rather than as the objective, inferring the true requirements from the instruction and the environment and satisfying them even when the user is not yet aware of all of them. We call this ability \textbf{outcome alignment}. In this work, we construct \textbf{TiGanEval}, a benchmark of software engineering and research tasks. In every task the user genuinely lacks part of the requirements, each requirement stays discoverable from the code and runtime behaviour of the environment, and requirements the agent infers are separated from those the user adds after observing delivered work. These properties together make outcome alignment measurable. Evaluating eight frontier models including GPT-5.6-Sol, we find outcome alignment broadly missing: the models lose 53 to 67 points of success rate when moving from the full specification to the partial instruction, and consequence feedback across four submissions does not close the gap. Furthermore, a plan-mode harness cannot compensate for the missing ability of the model itself, which places outcome alignment among the abilities the next generation of models should be developed for.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑