While LLMs have accelerated scientific code generation, comprehensively evaluating generated code remains challenging. Many benchmarks emphasize functional correctness or task completion, which is insufficient for code built on production HPC libraries, where solver selection, API conventions, memory management, parallel awareness, and performance also matter. We introduce PETSCAgent-Bench, a multidimensional benchmark and agent-based framework for assessing whether AI-generated scientific code uses a production HPC library as an expert would. A tool-augmented evaluator compiles, executes, and measures code and combines deterministic checks with LLM-based assessments in a 14-evaluator pipeline spanning five categories: correctness, performance, code quality, algorithmic appropriateness, and library-specific conventions. A2A and MCP enable black-box evaluation of compatible coding agents. Across realistic PETSc problems, frontier models generate readable, well-structured code but struggle with correctness on challenging problems and with library-specific conventions even when code compiles and runs---limitations that conventional pass/fail evaluation does not capture.
Positioning manuscripts in the scientific landscape with agentic AI
University of Pennsylvania(宾夕法尼亚大学);University of Illinois at Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);University of North Carolina at Chapel Hill(北卡罗来纳大学教堂山分校);Stanford University(斯坦福大学);Yale University(耶鲁大学)
Agentic programs: an emerging form of scientific software in computational materials science
Research Institute of Advanced Materials, Seoul National University(首尔国立大学先进材料研究所);Department of Materials Science and Engineering, Seoul National University(首尔国立大学材料科学与工程系);Center for AI and Natural Sciences, Korea Institute for Advanced Study(韩国高等研究院人工智能与自然科学中心)
PaperArena: An Evaluation Benchmark for Tool-Augmented Agentic Reasoning on Scientific Literature
State Key Laboratory of Cognitive Intelligence, University of Science and Technology of China(认知智能国家重点实验室,中国科学技术大学);Artificial Intelligence Research Institute, iFLYTEK Co., Ltd(人工智能研究院,iFLYTEK有限公司)
Bohrium + SciMaster: Building the Infrastructure and Ecosystem for Agentic Science at Scale
DP Technology Beijing China(北京DP技术有限公司);AI for Science Institute Beijing China(北京AI for Science研究院);Shanghai Jiao Tong University Shanghai China(上海交通大学);Beihang University Beijing China(北京航空航天大学);Peking University Beijing China(北京大学);Institute of Theoretical Physics Chinese Academy of Sciences Beijing China(中国科学院理论物理研究所);Shanghai Innovation Institute Shanghai China(上海创新研究院);East China Normal University Shanghai China(华东师范大学);Zhongguancun Academy Beijing China(中关村学院);University of Science and Technology of China Hefei China(中国科学技术大学);Tongji University Shanghai China(同济大学);The Hong Kong University of Science and Technology Hong Kong China(香港科技大学)
Agentic Publication Protocol: An Attempt to Modernize Scientific Publication
Max-Planck-Institut für Quantenoptik(马克斯·普朗克量子光学研究所);Munich Center for Quantum Science and Technology(慕尼黑量子科学与技术中心);Leinweber Institute for Theoretical Physics(莱因韦伯理论物理研究所)
KISS - Knowledge Infrastructure for Scientific Simulation: A Scaffolding for Agentic Earth Science
State Key Laboratory of Water Disaster Prevention, Hohai University, Nanjing(水利灾害预防国家重点实验室,河海大学,南京);Yangtze Institute for Conservation and Development, Hohai University, Nanjing(长江保护与发展研究院,河海大学,南京);Department of Bioresource Engineering, McGill University, Sainte-Anne-de-Bellevue, Quebec, Canada(生物资源工程系,麦吉尔大学,圣安妮-德-贝尔贝夫,魁北克,加拿大);Ottawa Research and Development Centre, Agriculture & Agri-Food Canada, Ottawa, Ontario, K1A 0C6, Canada(渥太华研发中心,加拿大农业与食品部,渥太华,安大略,K1A 0C6,加拿大);College of Water Conservancy and Hydropower Engineering, Hohai University, Nanjing(水利水电工程学院,河海大学,南京);Meta Platforms Inc.(Meta平台公司);Nanjing Hydraulic Research Institute, Nanjing(南京水利研究院)
Plausible but Wrong: A case study on Agentic Failures in Astrophysical Workflows
Bonn-Aachen International Center for Information Technology, University of Bonn, Germany(波恩-阿森国际信息科技中心,波恩大学,德国);Lamarr Institute for Machine Learning and Artificial Intelligence, Germany(拉马尔机器学习与人工智能研究所,德国)
FlyAOC: Evaluating Agentic Ontology Curation of Drosophila Scientific Knowledge Bases
University of Michigan(密歇根大学);University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);University of California, Berkeley(加州大学伯克利分校);University of Washington(华盛顿大学)
LQCDMaster: Agentic Scientific Computing for Lattice Quantum Chromodynamics Research
School of Physics and Astronomy(物理与天文学院);Tsung-Dao Lee Institute(李政道研究所);State Key Laboratory of Dark Matter Physics(暗物质物理国家重点实验室);School of Artificial Intelligence(人工智能学院);Zhiyuan College(智远学院);School of Physics(物理学院);SciLand;Institute of Quantum Matter(量子物质研究所);Department of Physics(物理系)
Reasoning With a Star: A Heliophysics Dataset and Benchmark for Agentic Scientific Reasoning
Frontier Development Lab(前沿发展实验室);Department of Mechanical and Aerospace Engineering, UCLA(机械与航空航天工程系,加州大学洛杉矶分校);Trillium Technologies Inc.(Trillium技术公司);Department of Engineering, University of Cambridge(工程系,剑桥大学)
El Agente Potente: High-Throughput Agentic Atomistic Simulations
University of Toronto(多伦多大学);Vector Institute for Artificial Intelligence(向量人工智能研究所);Acceleration Consortium(加速联盟);Canadian Institute for Advanced Research (CIFAR)(加拿大高等研究院(CIFAR));NVIDIA(英伟达)
Open-Source Agentic Hybrid RAG Framework for Scientific Literature Review
Department of Surgery, Department of Anesthesiology, Duke University School of Medicine Durham North Carolina United States;Faculty of Technology, University of Brasilia Brasilia Federal District Brazil;Indian Institute of Technology Goa Goa India;Birla Institute of Technology \& Science Pilani Hyderabad India;Department of Electrical;Computer Engineering, Duke University Pratt School of Engineering Department of Surgery, Department of Anesthesiology, Duke University School of Medicine Durham North Carolina United States;Department of Surgery, Department of Anesthesiology, Duke University School of Medicine;Faculty of Technology, University of Brasilia;Indian Institute of Technology Goa;Birla Institute of Technology \& Science Pilani;Computer Engineering, Duke University Pratt School of Engineering
VASP Agent: An Agentic Framework for Autonomous First-principles Calculations
Shanghai Artificial Intelligence Laboratory(上海人工智能实验室);Department of Computer Science and Technology(计算机科学与技术系);School of Physical Science and Technology(物理科学与技术学院);Department of Automation(自动化系);Beijing National Research Center for Information Science and Technology (BNRist)(北京信息科学与技术国家研究中心);School of Chemistry and Chemical Engineering(化学与化工学院);Key Laboratory of Computational Physical Sciences (Ministry of Education)(计算物理科学重点实验室);The Chinese University of Hong Kong(香港中文大学);Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences(中国科学院深圳先进技术研究所)