arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2602.11409cs.AI
收藏

TRACER:轨迹风险聚合用于代理推理中的关键事件

TRACER: Trajectory Risk Aggregation for Critical Episodes in Agentic Reasoning

University of Illinois at Chicago(伊利诺伊大学芝加哥分校);AI Labs at Capital One(Capital One 人工智能实验室)

Sina Tayebati, Divake Kumar, Nastaran Darabi, Davide Ettori, Ranganath Krishnan, Amit Ranjan Trivedi

AI总结:

TRACER通过轨迹级不确定性度量提升复杂对话工具使用中的不确定性检测精度。

摘要:

在现实世界中,为AI代理估计多轮工具使用交互中的人类不确定性是困难的,因为失败常常由稀疏的关键事件(例如循环、不一致的工具使用或用户-代理协调失误)触发,即使局部生成看起来自信。现有不确定性代理专注于单次文本生成,因此忽略了这些轨迹级崩溃信号。我们引入TRACER,一种用于双控工具-代理-用户交互的轨迹级不确定性度量。TRACER结合了内容感知的惊奇度、情境感知信号、语义和词汇重复,以及工具相关的连贯性缺口,并通过尾部聚焦的风险函数和MAX复合步风险进行聚合,以揭示决定性异常。我们通过τ²-bench评估TRACER,预测任务失败和选择性任务执行。为此,TRACER在基线模型上将AUROC提高了高达37.1%,将AUARC提高了高达55%,使在复杂对话工具使用设置中更早、更准确地检测不确定性成为可能。我们的代码和基准在https://github.com/sinatayebati/agent-tracer上可用。

英文摘要:

Estimating uncertainty for AI agents in real-world multi-turn tool-using interaction with humans is difficult because failures are often triggered by sparse critical episodes (e.g., looping, incoherent tool use, or user-agent miscoordination) even when local generation appears confident. Existing uncertainty proxies focus on single-shot text generation and therefore miss these trajectory-level breakdown signals. We introduce TRACER, a trajectory-level uncertainty metric for dual-control Tool-Agent-User interaction. TRACER combines content-aware surprisal with situational-awareness signals, semantic and lexical repetition, and tool-grounded coherence gaps, and aggregates them using a tail-focused risk functional with a MAX-composite step risk to surface decisive anomalies. We evaluate TRACER on $τ^2$-bench by predicting task failure and selective task execution. To this end, TRACER improves AUROC by up to 37.1% and AUARC by up to 55% over baselines, enabling earlier and more accurate detection of uncertainty in complex conversational tool-use settings. Our code and benchmark are available at https://github.com/sinatayebati/agent-tracer.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑