AI agents for autonomous scientific discovery must decide which experiments to run under a limited budget. Most benchmarks for science agents provide fixed datasets, while others that support active experimentation largely rely on simulation. We introduce Autonomous Science Bench, an initial release comprising six budgeted design-build-test-learn campaigns grounded in published experiments across biology, chemistry, and materials science. Agents select batches of designs, receive measurements from a lab service, and use the results to guide subsequent experiments before delivering a predictive model. Success requires passing two calibrated gates: a discovery gate, which evaluates the designs the agent measured, and a learning gate, which evaluates its model's ranking of held-out designs. In a pilot evaluation of ten agent-model configurations and 180 trials, the best configuration succeeds in 72% of trials, while the hardest campaign is solved in only five of 30 attempts. The two gates reveal different weaknesses: some agents build accurate rankings but find too few promising designs, while others find promising designs but fail to predict which unseen designs will perform well. These differences are obscured by overall success rates alone. We release version 0.1 of the campaigns, lab runtime, and public leaderboard at https://asb.open-ssi.org/.
From Research Goals to Scientific Closure: Evaluating Agents Through Evidence-Supported Physics Research
Shanghai AI Laboratory(上海人工智能实验室);Shanghai Jiao Tong University(上海交通大学);The HongKong University of Science and Technology(Guangzhou)(香港科技大学(广州));Nanyang Technological University(南洋理工大学)
FlyAOC: Evaluating Agentic Ontology Curation of Drosophila Scientific Knowledge Bases
University of Michigan(密歇根大学);University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);University of California, Berkeley(加州大学伯克利分校);University of Washington(华盛顿大学)
ScienceBoard: Evaluating Multimodal Autonomous Agents in Realistic Scientific Workflows
The University of Hong Kong(香港大学);Shanghai AI Laboratory(上海人工智能实验室);Fudan University(复旦大学);Peking University(北京大学);Nanjing University(南京大学);East China Normal University(华东师范大学);Yale University(耶鲁大学)
Benchmarking AI Agents for Addressing Scientific Challenges Across Scales
Yale University(耶鲁大学);Broad Institute of MIT and Harvard(布罗德研究所);The Pennsylvania State University(宾夕法尼亚州立大学);Northeastern University(东北大学);Northwestern University(西北大学)
EAIRA: Establishing a Methodology for Evaluating AI Models as Scientific Research Assistants
Argonne National Laboratory(阿贡国家实验室);The University of Chicago(芝加哥大学);University of Pennsylvania(宾夕法尼亚大学);Lawrence Livermore National Laboratory(劳伦斯利弗莫尔国家实验室);The Ohio State University(俄亥俄州立大学);Rochester Institute of Technology(罗切斯特理工学院);Massachusetts Institute of Technology(麻省理工学院)
PersonaScienceBench: Evaluating Scientific Agents with Diverse Simulated Researchers
University of Chicago(芝加哥大学);University of Illinois Chicago(芝加哥伊利诺伊大学);Harvard University(哈佛大学);Boston University(波士顿大学);Arizona State University(亚利桑那州立大学);Yale University(耶鲁大学);University of Southern California(南加利福尼亚大学);The University of Texas at Dallas(德克萨斯大学达拉斯分校);University of California, Los Angeles(加利福尼亚大学洛杉矶分校);Texas A&M University(德克萨斯农工大学);Brown University(布朗大学);University of Toronto(多伦多大学);Northeastern University(东北大学);Stanford University(斯坦福大学);University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);Purdue University(普渡大学);The Ohio State University(俄亥俄州立大学);MatrAIx
Jev for Scientific Decisions: Evaluating Semantic Choices and Their Consequences
Johns Hopkins University(约翰霍普金斯大学);Columbia University(哥伦比亚大学);The Hong Kong Polytechnic University(香港理工大学);The Pennsylvania State University(宾夕法尼亚州立大学)
Evaluating LLMs' Divergent Thinking Capabilities for Scientific Idea Generation with Minimal Context
Gaoling School of Artificial Intelligence, Renmin University of China(中国人民大学人工智能学院);College of Computer Science and Technology, Zhejiang University(浙江大学计算机科学与技术学院);Bank of China(中国银行);School of Engineering Science, University of Chinese Academy of Sciences(中国科学院大学工程科学学院);State Key Laboratory of Nonlinear Mechanics, Institute of Mechanics, Chinese Academy of Sciences(中国科学院力学研究所非线性力学重点实验室)
The Marathon of Scientific Reasoning: Robustness of Scientific Agents to Perturbations in Multi-Turn Interactions
Inria(法国国家信息与自动化研究所);Agency for Science, Technology and Research (A*STAR)(新加坡科技研究局);Zhejiang Key Laboratory of Artificial Intelligence of Things (AIoT) Network and Data Security(浙江省人工智能物联网网络与数据安全重点实验室)
EarthSE: A Benchmark for Evaluating Earth Scientific Exploration Capability of LLMs
Shanghai Jiao Tong University(上海交通大学);Shanghai Artificial Intelligence Laboratory(上海人工智能实验室);The Hong Kong Polytechnic University(香港理工大学);The Chinese University of Hong Kong(香港中文大学)
MSEarth: A Multimodal Scientific Dataset and Benchmark for Phenomena Uncovering in Earth Science
The Hong Kong Polytechnic University(香港理工大学);Shanghai Artificial Intelligence Laboratory(上海人工智能实验室);Shanghai Jiao Tong University(上海交通大学);The Chinese University of Hong Kong(香港中文大学)
SciVisAgentBench: A Benchmark for Evaluating Scientific Data Analysis and Visualization Agents
University of Notre Dame(圣母大学);Lawrence Livermore National Laboratory(劳伦斯利弗莫尔国家实验室);University of Utah(犹他大学);University of Nebraska–Lincoln(内布拉斯加大学林肯分校);The Ohio State University(俄亥俄州立大学);Argonne National Laboratory(阿贡国家实验室);Anthropic PBC
SciHorizon-eLab: An Agentic Protocol-to-Task Compiler for Scalable Benchmarking of Scientific Embodied Agents
Data Intelligence for Scientific Innovation Lab, Computer Network Information Center, Chinese Academy of Sciences(中国科学院计算机网络信息中心科学智能创新数据实验室);Beihang University(北京航空航天大学)