CatchBench: When Can an Agent Failure Be Caught?
CatchBench:智能体故障何时能被检测?
机构 * University of Southern California(南加州大学)
AI总结 CatchBench是首个在同一任务-方法接口下对智能体运行前、运行中、完成后三种信息状态进行评分的基准,涵盖多类模型与配置,揭示基准分数需结合标签过程才可解释。
Comments 39 pages, 6 figures, 21 tables. Work in progress. Code and data: this https URL (https://github.com/yzhao062/catchbench)