Recent advancements in Large Language Model (LLM) agents have demonstrated remarkable potential in automatic knowledge discovery. However, rigorously evaluating an AI's capacity for knowledge discovery remains a critical challenge. Existing benchmarks predominantly rely on static datasets, leading to inevitable data contamination where models have likely seen the evaluation knowledge during training. Furthermore, the rapid release cycles of modern LLMs render static benchmarks quickly outdated, failing to assess the ability to discover truly new knowledge. To address these limitations, we propose DBench-Bio, a dynamic and fully automated benchmark designed to evaluate AI's biological knowledge discovery ability. DBench-Bio employs a three-stage pipeline: (1) data acquisition of rigorous, authoritative paper abstracts; (2) QA extraction utilizing LLMs to synthesize scientific hypothesis questions and corresponding discovery answers; and (3) QA filter to ensure quality based on relevance, clarity, and centrality. We instantiate this pipeline to construct a monthly-updated benchmark covering 12 biomedical sub-domains. Extensive evaluations of SOTA models reveal current limitations in discovering new knowledge. Our work provides the first dynamic, automatic framework for assessing the new knowledge discovery capabilities of AI systems, establishing a living, evolving resource for AI research community to catalyze the development of knowledge discovery.
Automatic Ordinary Differential Equations Discovery For Biological Systems Using Large Language Model Powered Agentic System
Weizmann Institute of Science（魏茨曼科学研究所）；Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）；University of Haifa（海法大学）；Jonkoping University（延雪平大学）
A Multimodal Foundation Model of Spatial Transcriptomics and Histology for Biological Discovery and Clinical Prediction
Department of Radiation Oncology, Stanford University School of Medicine（斯坦福大学医学院放射肿瘤学系）；Department of Statistics and Data Science, Yale University（耶鲁大学统计与数据科学系）；Department of Medicine (Oncology), Stanford University School of Medicine（斯坦福大学医学院医学系（肿瘤学））；Department of Pathology, Stanford University School of Medicine（斯坦福大学医学院病理学系）；Department of Neurosurgery, Stanford University School of Medicine（斯坦福大学医学院神经外科学系）；Stanford Institute for Human-Centered Artificial Intelligence（斯坦福大学以人为本人工智能研究所）；Perelman School of Medicine at the University of Pennsylvania（宾夕法尼亚大学佩雷尔曼医学院）
An LLM-based Knowledge Synthesis and Scientific Reasoning Framework for Biomedical Discovery
Idiap Research Institute（伊迪亚普研究所）；National Biomarker Centre（国家生物标志物中心）；CRUK-MI（英国癌症研究中心曼彻斯特研究所）；Univ. of Manchester（曼彻斯特大学）；Department of Computer Science, Univ. of Manchester（曼彻斯特大学计算机科学系）
Allen Institute for AI（艾伦人工智能研究所）；Brandeis University（布兰迪斯大学）；Tohoku University（东北大学）；Paul G. Allen School of Computer Science & Engineering（保罗·G·艾伦计算机科学与工程学院）；University of Colorado Boulder（科罗拉多大学博尔德分校）
