Agent harnesses enable large language models to act through tools, observe the results, and iteratively work toward a task goal. Verification checks and revises agents' work to improve output quality. Yet there is limited guidance on how harnesses should organize this effort. We study verification as an axis of test-time scaling by examining when to intervene, who checks the work, and how checking leads to revision. Our framework spans four layers of the agent lifecycle: initial instructions, mid-turn reminders, completion gates, and independent review. We compare 17 designs across seven frontier coding models on Terminal-Bench (TB) and SWE-bench Verified (SWE), and introduce VisualBench for expert evaluation of visual and interactive artifacts. Initial instructions and execution-time reminders yield modest, model-dependent average gains on the coding benchmarks, without substantially increasing average output token use. Completion gates can yield larger gains but may substantially increase cost. Independent review by a critic offers a more favorable accuracy-cost tradeoff, improving average accuracy by 5.9-9.5 percentage points on TB and 3.9-6.6 on SWE at 1.13-1.25 and 1.00-1.04 times baseline cost, respectively. On VisualBench, independent review achieves net preferences of 13.2-23.8 points over the baseline across its four designs, exceeding the evaluated L0-L2 designs. Additional review rounds or independent critics further improve average TB accuracy by up to 3.5 percentage points. Together, these results provide empirical guidance for verification design in agent harnesses and support independent review as a practical axis of test-time scaling.
When to Rethink: Learning Multi-Perspective Self-Verification for Vision-Language Models
University of Leicester(莱斯特大学);Nanjing University of Posts and Telecommunications(南京邮电大学);University of North Carolina at Chapel Hill(北卡罗来纳大学教堂山分校);Amazon(亚马逊);University of Exeter(埃克塞特大学);University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);Meta AI;Eindhoven University of Technology(埃因霍温理工大学)
Beyond Model Collapse: Scaling Up with Synthesized Data Requires Verification
Meta FAIR;Center for Data Science, New York University(纽约大学数据科学中心);Courant Institue of Mathematical Sciences, New York University(纽约大学柯朗数学科学研究所);School of Mathematical Sciences, Peking University(北京大学数学科学学院)
Agents Are Systems, Not Models: Rethinking Agentic Evaluation
Technical University of Munich(慕尼黑工业大学);Helmholtz Munich(亥姆霍兹慕尼黑);Inria(法国国家信息与自动化研究所);École normale supérieure(巴黎高等师范学院);CNRS(法国国家科学研究中心);PSL Research University(巴黎文理研究大学)
Learning to Evaluate Before Improving: Automatic Rubric Induction for Automatic Research Agents
Zhejiang University(浙江大学);University of Electronic Science and Technology of China(电子科技大学);Beijing University of Posts and Telecommunications(北京邮电大学);Zhejiang University of Technology(浙江工业大学)
Hardware Phi-1.5B: A Large Language Model Encodes Hardware Domain Specific Knowledge
Kansas State University(堪萨斯州立大学);University of Science and Technology of China(中国科学技术大学);Michigan Technological University(密歇根理工大学);Washington University in St. Louis(圣路易斯华盛顿大学);Silicon Assurance(硅保障公司);Tianjin University(天津大学)
Trial by Trace: Benchmarking AI Agent Harnesses for Reliability and Failure Attribution
Institute of Automation, CAS(中国科学院自动化研究所);Zhongguancun Academy(中关村学院);Institute of Information Engineering, CAS(中国科学院信息工程研究所);Tsinghua University(清华大学);Beijing Institute of Technology(北京理工大学)
A Theory of Reliable Self-Evolution for Agent Harnesses
University of Science and Technology of China(中国科学技术大学);Hong Kong Generative AI Research & Development Center(香港生成式人工智能研发中心);The Hong Kong University of Science and Technology(香港科技大学);Hong Kong Baptist University(香港浸会大学)