发表机构
UCAS; Institute of Computing Technology, CAS; DataArc Tech Ltd.; IDEA Research, International Digital Economy Academy; Jiangnan University; The Hong Kong University of Science and Technology (Guangzhou)（中国科学院大学; 中国科学院计算技术研究所; DataArc科技有限公司; 国际数字经济学院IDEA研究院; 江南大学; 香港科技大学（广州））
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
针对LLM智能体长时任务评估的三大局限，提出动态分阶段轨迹评估框架DynSTEER，通过里程碑分段、路径容忍图和自适应多级评判，提升区分度85.2%并节省34.51%执行步骤。
AI 中文摘要
大型语言模型智能体越来越多地被部署用于长时间跨度的任务执行。然而，当前的评估范式面临三大主要局限：仅基于最终结果的评估忽略了中间过程，并且难以高效、准确地定位错误；单一参考匹配会惩罚有效的替代解决方案路径；事后轨迹评判成本高昂，且无法提前终止失败的运行。为了解决这些问题，我们提出了DynSTEER，一个面向智能体的动态分阶段轨迹评估框架。DynSTEER将轨迹展开划分为以关键已完成动作锚定的阶段，在提供充分上下文的同时聚焦于关键里程碑的评估，并支持针对性的策略调整。它从公开任务视图中构建一个路径容忍的里程碑图，以尊重多样化的合法策略而不泄露真实标签。此外，它自适应地将评估查询路由到多层级评判器，并在线终止不可恢复的执行以遏制资源浪费。实验表明，与原生评估相比，DynSTEER在LLM智能体上的评估区分度提高了85.2%，所有模型对均具有统计显著性差异，并在失败的轨迹展开上节省了34.51%的执行步骤。
英文摘要
Large language model agents are increasingly deployed for long-horizon task execution, raising a central granularity question for trajectory evaluation: whole-trajectory verification is too coarse to capture concrete failures and their associated evidence in long trajectories, while atomic-step scoring is too fine-grained, noise-sensitive, and computationally expensive. This granularity gap makes a single-reference trajectory paradigm inadequate for assessing the rich space of valid agent execution paths and delays timely feedback and early stopping in long-horizon tasks. To address these issues, we propose DynSTEER, a dynamic stage-wise framework for agent trajectory evaluation. DynSTEER bridges the granularity gap through stage-wise dynamic evaluation that segments rollouts at key execution nodes and adapts its multi-level review strategy based on stage-level results; it compiles a path-tolerant milestone graph from available task inputs to preserve diverse legal paths without reference leakage; and it supports terminating unrecoverable agent executions to curb resource waste. Experimental results show that DynSTEER improves evaluation discriminability by over 85\% compared with whole-trajectory evaluation and saves 17.74\% of execution steps. The code is available at https://github.com/zhichao-stone/DynSTEER
