AgentEval:基于证据约束的有状态智能体评估
AgentEval: Evidence-Bounded Assessment of Stateful Agents
AI总结:
提出AgentEval系统,通过保留合同约束的评估记录区分智能体行为与证据有效性,经多组实验验证其在智能体义务判定、证据检查及报告复用中的有效性。
摘要:
仅任务完成情况无法判定有状态智能体是否履行了其义务,且已记录的判断无法保证其支撑证据仍然有效。我们提出AgentEval,这一系统会在工具执行、状态与事件捕获、报告资格认定及案例维护过程中保留合同约束的评估记录。它区分了智能体的行为、证据所确立的内容以及结论是否适用于当前使用。在32次工具世界管理操作中,独立检查对全部30个完整场景达成一致,区分了已履行、已违反及未解决的义务;2次中断的管理操作被保留。对48个证据视图的受控干预暴露了仅哈希检查未发现的36个绑定故障,且未改变智能体行为。Scenario Studio将经审查的案例与12个两轮报告关联,这些报告包含24个经独立检查的答案。针对192次保留执行的单独公共基准研究,揭示了在更广泛的任务失败中存在10个已完成的返回子目标,并分离出缺失证据的影响。这些研究表明,可执行的评估记录如何使范围、证据支持及报告可在智能体评估工作流中重复检查。
英文摘要:
Task completion alone does not determine whether a stateful agent satisfied its obligations, and a recorded judgment does not guarantee that its supporting evidence remains valid. We present AgentEval, a system that carries a contract-bound assessment record through tool execution, state and event capture, report qualification, and case maintenance. It separates what the agent did, what the evidence establishes, and whether the conclusion is supported for present use. On 32 tool-world administrations, independent checks agree on all 30 complete episodes, distinguishing satisfied, violated, and unresolved obligations; two interrupted administrations are retained. Controlled interventions on 48 evidence views expose 36 binding faults missed by hash-only checks without changing agent behavior. Scenario Studio connects reviewed cases to 12 two-turn reports with 24 independently checked answers. A separate public-benchmark study on 192 held-out executions reveals ten completed return subgoals within broader task failures and isolates the effect of missing evidence. These studies show how an executable assessment record makes scope, evidential support, and report reuse inspectable across an agent evaluation workflow.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。