基于智能体序贯证伪的自动化假设验证
Automated Hypothesis Validation with Agentic Sequential Falsifications
AI总结：
针对LLM生成假设难以人工验证的问题，提出基于波普尔证伪原则的智能体框架Popper，通过LLM智能体设计执行序贯证伪实验并严格控制I类错误，在多领域实现稳健性能，验证生物假设时耗时仅为人类的1/10。
摘要：
假设是信息获取、决策和发现的核心。然而，许多现实世界的假设是抽象的高层陈述，难以直接验证。大型语言模型（LLMs）生成假设的兴起进一步加剧了这一挑战，LLMs易产生幻觉且生成的假设数量庞大，使得人工验证不切实际。本文提出Popper，一个用于自由形式假设严格自动化验证的智能体框架。受卡尔·波普尔证伪原则的指导，Popper使用LLM智能体设计并执行针对假设可测量含义的证伪实验来验证假设。一种新颖的序贯测试框架确保了严格的I类错误控制，同时从现有数据或新执行的程序中主动收集来自不同观测的证据。我们在生物学、经济学和社会学等六个领域展示了Popper。Popper实现了稳健的错误控制、高功效和可扩展性。此外，与人类科学家相比，Popper在验证复杂生物学假设时达到了相当的性能，同时将时间减少了10倍，为假设验证提供了一种可扩展、严谨的解决方案。
英文摘要：
Hypotheses are central to information acquisition, decision-making, and discovery. However, many real-world hypotheses are abstract, high-level statements that are difficult to validate directly. This challenge is further intensified by the rise of hypothesis generation from Large Language Models (LLMs), which are prone to hallucination and produce hypotheses in volumes that make manual validation impractical. Here we propose Popper, an agentic framework for rigorous automated validation of free-form hypotheses. Guided by Karl Popper's principle of falsification, Popper validates a hypothesis using LLM agents that design and execute falsification experiments targeting its measurable implications. A novel sequential testing framework ensures strict Type-I error control while actively gathering evidence from diverse observations, whether drawn from existing data or newly conducted procedures. We demonstrate Popper on six domains including biology, economics, and sociology. Popper delivers robust error control, high power, and scalability. Furthermore, compared to human scientists, Popper achieved comparable performance in validating complex biological hypotheses while reducing time by 10 folds, providing a scalable, rigorous solution for hypothesis validation.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
