The success score of a tool agent reflects both its model and the budgets of the harness that runs it. We present ToolBottleneckBench, a benchmark that measures tool-agent performance under controlled pressures and fixed harness budgets. Synthetic enterprise tasks stay fixed while one pressure varies: the registered tool catalog, the visible tool menu, the certified workflow, or the encoding of returned records. An exact grader records the first unmet obligation of each failed episode and the harness limit, if any, that ended it. Across 21 model and harness configurations, the visible menu is the only one of the three pressures that act on the model with a large penalty, and this penalty tracks truncation at the completion cap. Raising the cap sixteenfold removes most of the penalty in the truncation-heavy configurations. Most failures end at a harness limit, and the failure record identifies the assigned pressure no more reliably than a count of tool calls. The repairs that help change which tools reach the model or who issues the calls, and only two repairs, neither improving model behavior, beat every configuration switch. We recommend reporting pressure effects with the harness budgets under which they were measured. We release the benchmark at https://github.com/zichuanxu/tool-bottleneck-bench.
CLOSER-Bench: Evaluating Budgeted Cross-Stage Design Closure for Hardware Agents
Institute of Computing Technology, Chinese Academy of Sciences(中国科学院计算技术研究所);University of the Chinese Academy of Sciences(中国科学院大学);School of Advanced Interdisciplinary Sciences(先进交叉学科研究院)
EvoCowork: A Benchmark for Real-World Office Tasks
Peking University(北京大学);Sun Yat-sen University(中山大学);Shanghai Jiao Tong University(上海交通大学);Minzu University of China(中央民族大学);Beihang University(北京航空航天大学);University of Science and Technology of China(中国科学技术大学)
EvoPolicyGym: Evaluating Autonomous Policy Evolution in Interactive Environments
University of Science and Technology of China(中国科学技术大学);The Chinese University of Hong Kong(香港中文大学);University of Macau(澳门大学);Tsinghua University(清华大学);Zhejiang University(浙江大学);Soochow University(苏州大学);Brown University(布朗大学);Shanghai Jiao Tong University(上海交通大学)
AutoLab: Can Frontier Models Solve Long-Horizon Auto Research and Engineering Tasks?
MIT(麻省理工学院);Stanford University(斯坦福大学);University of California, Berkeley(加州大学伯克利分校);University of California, Los Angeles(加州大学洛杉矶分校);University of California, San Diego(加州大学圣地亚哥分校);University of Washington(华盛顿大学);University of Toronto(多伦多大学);University of Michigan(密歇根大学);National University of Singapore(新加坡国立大学);University of Tokyo(东京大学)
Judged Useless, Queried Anyway: Tool-Using Agents Rarely Turn Their Own Evidence Judgments into Stopping Decisions
Nanyang Technological University(南洋理工大学);National University of Singapore(新加坡国立大学);CFAR, Agency for Science, Technology and Research(新加坡科技研究局高等研究院);UNC-Chapel Hill(北卡罗来纳大学教堂山分校);Carnegie Mellon University(卡内基梅隆大学);IHPC, Agency for Science, Technology and Research(新加坡科技研究局高性能计算研究院)