发表机构
The Broad Institute of MIT and Harvard; Harvard Medical School; The Jackson Laboratory; Sutter Hill Ventures; Massachusetts Institute of Technology; Howard Hughes Medical Institute; The Wyss Institute for Biologically Inspired Engineering at Harvard University; Yale School of Medicine; Harvard T.H. Chan School of Public Health; Harvard University（麻省理工学院与哈佛大学博德研究所; 哈佛医学院; 杰克逊实验室; 萨特山风投; 麻省理工学院; 霍华德·休斯医学研究所; 哈佛大学维斯生物启发工程研究所; 耶鲁医学院; 哈佛陈曾熙公共卫生学院; 哈佛大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
研究人员提出科学沙箱框架，在调控基因组学和蛋白质适应性预测场景中评估前沿AI智能体的科学能力，发现其常不理解系统规则却能优化指标，该框架可衡量科学能力前沿并提供研究环境。
AI 中文摘要
科学进步不仅依赖于找到解决方案，还依赖于理解这些方案为何有效的规则，并利用这种理解设计更好的实验。我们引入科学沙箱（science sandboxes），这是一种通过实验、反馈和假设修正的循环来研究AI智能体此能力的框架。科学沙箱允许智能体以不同方式查询自然世界，从“湿”物理实验，到基于经验数据训练的“半湿”预测模型，再到“干”的自创规则。通过建立通用实验循环和其中评估智能体的协议，科学沙箱可评估特定指标上的定量性能，以及跨经验可验证性范围的定性科学推理。在此，我们在两个生物学场景中实例化该框架：调控基因组学模型和蛋白质适应性预测，并研究前沿智能体的能力。在这些场景中，我们可观察到智能体何时在不理解系统底层规则的情况下成功优化定量指标；尤其当它们遇到规则超出熟悉生物学先验的系统时，其科学推理会恶化。通过凸显此类失败模式，科学沙箱使科学能力的前沿可衡量，并提供受控环境以研究并最终拓展该能力。
英文摘要
Scientific progress depends not only on finding solutions, but on learning the rules that explain why they work and using that understanding to design better experiments. We introduce science sandboxes, a framework for studying this capability in AI agents through repeated cycles of experimentation, feedback, and hypothesis revision. Science sandboxes invite an agent to query the natural world in different ways, ranging from "wet" physical experiments, to "damp" predictive models trained on empirical data, to "dry" invented rules. By establishing a common experimental loop and a protocol for evaluating agents within it, science sandboxes allow assessment of both quantitative performance on specific metrics and qualitative scientific reasoning, across a spectrum of empirical verifiability. Here, we instantiate this framework in two biological settings, models of regulatory genomics and protein fitness prediction, and examine the capabilities of frontier agents. Across these settings, we could see when agents successfully optimized a quantitative metric without understanding the rules underlying the system. In particular, their scientific reasoning deteriorated when they encountered systems whose rules fell outside familiar biological priors. By highlighting such failure modes, science sandboxes make the frontier of scientific capability measurable and provide a controlled setting in which to study and ultimately expand it.
Comments72 pages, 5 main figures, 3 tables, and 5 supplementary figures; includes supplementary agent instructions and harness
