PaperArena: An Evaluation Benchmark for Tool-Augmented Agentic Reasoning on Scientific Literature
State Key Laboratory of Cognitive Intelligence, University of Science and Technology of China(认知智能国家重点实验室,中国科学技术大学);Artificial Intelligence Research Institute, iFLYTEK Co., Ltd(人工智能研究院,iFLYTEK有限公司)
Daoyu Wang, Mingyue Cheng, Shuo Yu, Zirui Liu, Ze Guo, Xin Li, Qi Liu
Understanding and reasoning on the large-scale scientific literature is a crucial touchstone for large language model (LLM) based agents. However, existing works are mainly restricted to tool-free tasks within single papers, largely due to the lack of a benchmark that evaluates cross-paper reasoning and multi-tool orchestration in authentic research scenarios. In this work, we propose PaperArena, a benchmark to evaluate LLM-based agents on questions that require integrating information across multiple papers with the assistance of external tools. Given a research question, agents should formulate a reasoning plan, interact with multiple papers, and invoke appropriate tools to produce a well-grounded answer. To support standardized evaluation, we provide a platform for agent execution, offering a modular tool environment including multimodal parsing, context retrieval, and programmatic computation. Experiments reveal that even the leading LLM powering a well-established agentic workflow achieves merely 38.78% average accuracy, while on the hard subset, accuracy drops to only 18.47%. We also analyze reasoning traces and diagnose agent behavior, providing the community with insights to develop and evaluate more capable scientific agents.
Agentic Retrieval-Augmented Generation: A Survey on Agentic RAG
Cleveland State University(克利夫兰州立大学);Kent State University(肯特州立大学);Northeastern University(东北大学);Roux Institute at Northeastern University(东北大学鲁克斯研究所);University of Agder (UiA)(阿格德尔大学)
Reasoning RAG via System 1 or System 2: A Survey on Reasoning Agentic Retrieval-Augmented Generation for Industry Challenges
Beijing University of Posts and Telecommunications(北京邮电大学);University of Georgia(佐治亚大学);South China University of Technology(华南理工大学);Technical University of Munich, University of Cologne(慕尼黑技术大学、科隆大学);SenseTime Research(商汤科技研究院);Qingyuan Research Institute, Shanghai Jiaotong University(青原研究院,上海交通大学)
SoK: Agentic Retrieval-Augmented Generation (RAG): Taxonomy, Architectures, Evaluation, and Research Directions
University of North Dakota(北达科他大学);Youngstown State University(青年州大学);University of Toledo(托莱多大学);University of Missouri(密苏里大学);Tribhuvan University(特里布文大学)
Accelerating Manufacturing Scale-Up from Material Discovery Using Agentic Web Navigation and Retrieval-Augmented AI for Process Engineering Schematics Design
Grounded by Experience: Generative Healthcare Prediction Augmented with Hierarchical Agentic Retrieval
Department of Electronic and Computer Engineering, The Hong Kong University of Science and Technology(电子与计算机工程系,香港科技大学);College of Management and Economics, Laboratory of Computation and Analytics of Complex Management Systems (CACMS), Tianjin University(管理学院与经济学学院,复杂管理系统的计算与分析实验室,天津大学);Department of Computer Science and Engineering, The Hong Kong University of Science and Technology(计算机科学与工程系,香港科技大学)
APEX-Searcher: Refining Credit Assignment with Subgoaling for Agentic Retrieval-Augmented Generation
University of Chinese Academy of Sciences(中国科学院大学);MAIS, Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所MAIS部);Wenge Technology Co., Ltd(Wenger科技有限公司)
Forgotten in Weights, Recovered by Tools: Agentic Tool Unlearning for LLM Agents
The Chinese University of Hong Kong, Shenzhen(香港中文大学(深圳));University of Notre Dame(圣母大学);Johns Hopkins University(约翰斯·霍普金斯大学);Northwestern University(西北大学);MIT(麻省理工学院)
QUASAR: Quantum Assembly Code Generation Using Tool-Augmented LLMs via Agentic RL
Aalto University(阿alto大学);University of Helsinki(赫尔辛基大学);University of Liverpool(利物浦大学);University of Southampton(南安普顿大学);Technical University of Denmark(丹麦技术大学)
MAGMA: A Multi-Graph based Agentic Memory Architecture for AI Agents
Department of Computer Science, The University of Texas at Dallas(德克萨斯大学达拉斯分校计算机科学系);Department of Electrical and Computer Engineering, University of Florida(佛罗里达大学电气与计算机工程系)
Knowledge-augmented Agentic AI for Mental Health Medication Information Seeking
Department of Medicine and Therapeutics, The Chinese University of Hong Kong(香港中文大学内科及药物治疗学系);Department of Psychiatry, The Chinese University of Hong Kong(香港中文大学精神科学系);School of Control Science and Engineering, Shandong University(山东大学控制科学与工程学院);College of Information Science, University of Arizona(亚利桑那大学信息科学学院);Department of Medicine, Stanford University School of Medicine, Stanford University(斯坦福大学医学院医学系);Perelman School of Medicine, The University of Pennsylvania(宾夕法尼亚大学佩雷尔曼医学院);Department of Biostatistics, Vanderbilt University(范德堡大学生物统计学系);Li Ka Shing Institute of Health Sciences, Faculty of Medicine, The Chinese University of Hong Kong(香港中文大学医学院李嘉诚健康科学研究所)
Formal Foundations of Agentic Business Process Management
University of Oxford, UK(牛津大学,英国);SAP, Germany(德国SAP公司);Umeå University, Sweden(乌梅拉大学,瑞典);University of Mannheim, Germany(曼海姆大学,德国);Free University of Bozen-Bolzano, Italy(博兹纳自由大学,意大利)
RADIANT-LLM: an Agentic Retrieval Augmented Generation Framework for Reliable Decision Support in Safety-Critical Nuclear Engineering
Department of Nuclear Engineering, Texas A\&M University, College Station, TX, USA;College of Performance, Visualization;Fine Arts, Texas A\&M University, College Station, TX, USA
Agentic AI for Bilevel Long-Term Optimization of Policy-Driven Physical Layer Systems
Key Laboratory of EMW Information (MoE), College of Future Information Technology, Fudan University(复旦大学未来信息技术学院电磁波信息科学教育部重点实验室);James Watt School of Engineering, University of Glasgow(格拉斯哥大学詹姆斯·瓦特工程学院);School of Engineering, Edith Cowan University(埃迪斯科文大学工程学院);Information Systems Technology and Design Pillar, Singapore University of Technology and Design(新加坡科技设计大学信息系统技术与设计系)
Agentic Graph Retrieval-Augmented Generation for Auditable Commercial Registry Analysis
Computational Social Science ETH Zürich(计算社会科学 ETH 瑞士 Zurich);Computational Social Science, ETH Complexity Science Hub Zürich(计算社会科学 ETH 复杂性科学中心 Zurich);ETH Complexity Science Hub Zürich, Switzerland(ETH 复杂性科学中心 Zurich 瑞士);Vienna, Austria(维也纳 奥地利)