LLM agents have incredible potential for scientific discovery applications. However, the performance of LLM agents on real-world, small molecule drug design (SMDD) tasks across diverse chemistries and targets is unclear. Current evaluation methods are either ad hoc, too simple for real-world discovery, limited in scale, or restricted to single-turn question answering. In effort to standardize the evaluation of LLM agents on small molecule design, we introduce SMDD-Bench, a challenging, multi-turn, long-horizon agentic benchmark consisting of 502 guaranteed-solvable task instances spanning 5 task types: 2D Pharmacophore Identification, Interaction Point Discovery, Scaffold Hopping, Lead Optimization, and Fragment Assembly. SMDD-Bench tasks span a wide region of chemical space and involve 102 unique protein targets. Completely solving the benchmark would require having strong chemical and biological reasoning and 3D intuition, understanding specialized tool use, and displaying planning expertise over a limited number of oracle calls. We benchmark 7 frontier open and closed source LLMs and find even the most performant LLM, GPT5.4, solves only 40.2\% of tasks. We hope SMDD-Bench provides a standardized testbed to invigorate the field towards training and evaluating LLM agents for fully autonomous computational drug design. We host a public leaderboard at smddbench.com .
DrugPlayGround: Benchmarking Large Language Models and Embeddings for Drug Discovery
Yale University(耶鲁大学);Broad Institute of MIT and Harvard(麻省理工学院和哈佛大学博德研究所);The Chinese University of Hong Kong(香港中文大学);University of California, Berkeley(加州大学伯克利分校);UC Berkeley(加州大学伯克利分校)
Lived Experience Not Found: LLMs Struggle to Align with Experts on Addressing Adverse Drug Reactions from Psychiatric Medication Use
College of Computing, Georgia Institute of Technology(佐治亚理工学院计算学院);Hospital General Universitario Gregorio Marañón(格雷戈里奥·马拉尼翁大学综合医院);Hofstra University(霍夫斯特拉大学);Columbia University(哥伦比亚大学)
DrugOOD: Out-of-Distribution (OOD) Dataset Curator and Benchmark for AI-aided Drug Discovery -- A Focus on Affinity Prediction Problems with Noise Annotations
Tencent AI Lab(腾讯人工智能实验室);The University of Hong Kong(香港大学);Fudan University(复旦大学)
Benchmark on Drug Target Interaction Modeling from a Drug Structure Perspective
University of Minnesota(明尼苏达大学);University of California San Diego(加州大学圣地亚哥分校);UNC Chapel Hill(北卡罗来纳大学教堂山分校);North Carolina State University(北卡罗来纳州立大学)
DDIAgents: Mechanism-Conditioned Context Flow for Drug-Drug Interaction Prediction
Department of Electronic Engineering, Tsinghua University(清华大学电子工程系);Institute of Biomedical Engineering, University of Oxford(牛津大学生物医学工程研究所);Division of Emerging Interdisciplinary Areas, Hong Kong University of Science and Technology(香港科技大学新兴跨学科领域学部)
ADMEOOD: Out-of-Distribution Benchmark for Drug Property Prediction
College of Informatics, Huazhong Agricultural University(华中农业大学信息学院);Shenzhen Institute of Nutrition and Health, Huazhong Agricultural University(华中农业大学深圳营养与健康研究院)
ImDrug: A Benchmark for Deep Imbalanced Learning in AI-aided Drug Discovery
Tencent AI Lab(腾讯人工智能实验室);The Chinese University of Hong Kong(香港中文大学);Tsinghua University(清华大学);Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州));Hong Kong University of Science and Technology(香港科技大学);Renmin University of China(中国人民大学);University of Science and Technology of China(中国科学技术大学)
Evaluation Framework for AI-driven Molecular Design of Multi-target Drugs: Brain Diseases as a Case Study
Programa de Pós-graduação em Computação (PPGC)(计算机研究生项目(PPGC));Hub de Inovação em Inteligência Artificial (H2IA)(人工智能创新中心(H2IA));Omixlab, Centro de Desenvolvimento Tecnológico (CDTec), Universidade Federal de Pelotas (UFPel)(奥米克斯实验室、技术发展中心(CDTec)、佩洛塔斯联邦大学(UFPel))
Structure-based Drug Design Benchmark: Do 3D Methods Really Dominate?
Rensselaer Polytechnic Institute(伦斯勒理工学院);Stanford Medicine School(斯坦福医学院);Harvard Medical School(哈佛医学院);University of Science and Technology of China (USTC)(中国科学技术大学);The Ohio State University(俄亥俄州立大学)
CovDocker: Benchmarking Covalent Drug Design with Tasks, Datasets, and Solutions
Huazhong University of Science and Technology(华中科技大学);Microsoft Research AI for Science(微软研究院人工智能科学部);Shanghai Artificial Intelligence Laboratory(上海人工智能实验室)
Beyond Affinity: A Benchmark of 1D, 2D, and 3D Methods Reveals Critical Trade-offs in Structure-Based Drug Design
Department of Computer Science Rensselaer Polytechnic Institute(计算机科学系伦塞拉尔理工学院);Department of Computer Science and Engineering Lehigh University(计算机科学与工程系莱斯大学);Department of Computer Science University of Southern California(计算机科学系南加州大学);Stanford Medicine Department of Pathology Stanford University(斯坦福医学部病理学系斯坦福大学);Princeton University(普林斯顿大学);Harvard Medical School(哈佛医学院);State Key Laboratory for Novel Software Technology at Nanjing University(南京大学新型软件技术国家重点实验室);Department of Computer Science and Engineering The Chinese University of Hong Kong(计算机科学与工程系香港中文大学)
DrugAgent: Reliable Multi-Agent Integration of Conflicting Biomedical Evidence for Drug-Target Interaction Assessment
Department of Computer Science and Engineering, University of Minnesota(计算机科学与工程系,明尼苏达大学);Computational Biology Branch, National Library of Medicine(国家医学图书馆计算生物学分支);Khoury College of Computer Sciences, Northeastern University(东北大学计算机科学学院);State Key Laboratory for Novel Software Technology at Nanjing University, School of Computer Science, Nanjing University(南京大学新型软件技术国家重点实验室,南京大学计算机科学学院);Developmental Therapeutics Branch, National Cancer Institute(国家癌症研究所发育治疗分支)
MoleculeCLA: Rethinking Molecular Benchmark via Computational Ligand-Target Binding Analysis
Institute for AI Industry Research, Tsinghua University(清华大学人工智能产业研究院);College of Computer Science and Technology, Jilin University(吉林大学计算机科学与技术学院);Peking University(北京大学)
Contrastive Multi-Task Learning with Solvent-Aware Augmentation for Drug Discovery
Department of Health Technology and Informatics, The Hong Kong Polytechnic University(健康科技与信息学系,香港理工大学);Department of Nuclear Medicine and PET, Hong Kong Sanatorium and Hospital(核医学与PET部,香港疗养院及医院);School of Data Science, Lingnan University(数据科学学院,岭南大学)
UniMatch: Universal Matching from Atom to Task for Few-Shot Drug Discovery
Zhejiang University(浙江大学);Zhejiang Laboratory(浙江实验室);Shanghai Jiao Tong University(上海交通大学);University of Chinese Academy of Sciences(中国科学院大学);ZJU-Hangzhou Global Scientific and Technological Innovation Center(浙大杭州国际科创中心)
FusionDTI: Fine-grained Binding Discovery with Token-level Fusion for Drug-Target Interaction
School of Computing Science(计算科学学院);School of Cancer Sciences(癌症科学学院);Cancer Research UK Scotland Institute(英国癌症研究苏格兰研究所);University of Glasgow(格拉斯哥大学)
Towards Unified AI Drug Discovery with Multiple Knowledge Modalities
Institute for AI Industry Research (AIR), Tsinghua University(清华大学人工智能产业研究院(AIR));School of Software and Microelectronics, Peking University(北京大学软件与微电子学院)