Large Language Models (LLMs) generalize well across language tasks, but suffer from hallucinations and uninterpretability, making it difficult to assess their accuracy without ground-truth. Retrieval-Augmented Generation (RAG) models have been proposed to reduce hallucinations and provide provenance for how an answer was generated. Applying such models to the scientific literature may enable large-scale, systematic processing of scientific knowledge. We present PaperQA, a RAG agent for answering questions over the scientific literature. PaperQA is an agent that performs information retrieval across full-text scientific articles, assesses the relevance of sources and passages, and uses RAG to provide answers. Viewing this agent as a question answering model, we find it exceeds performance of existing LLMs and LLM agents on current science QA benchmarks. To push the field closer to how humans perform research on scientific literature, we also introduce LitQA, a more complex benchmark that requires retrieval and synthesis of information from full-text scientific papers across the literature. Finally, we demonstrate PaperQA's matches expert human researchers on LitQA.
SciAgent: Tool-augmented Language Models for Scientific Reasoning
Microsoft(微软公司);University of California, Santa Barbara(加利福尼亚大学圣巴巴拉分校);Tsinghua University(清华大学);Singapore Management University(新加坡管理大学);Nanyang Technological University(南洋理工大学)
OpenScholar: Synthesizing Scientific Literature with Retrieval-augmented LMs
University of Washington(华盛顿大学);Allen Institute for AI(艾伦人工智能研究所);University of Illinois, Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);Carnegie Mellon University(卡内基梅隆大学);Meta;University of North Carolina, Chapel Hill(北卡罗来纳大学教堂山分校);Stanford University(斯坦福大学)
SciRerankBench: Benchmarking Rerankers Towards Scientific Retrieval-Augmented Generation with LLMs
Computer Network Information Center, Chinese Academy of Sciences(中国科学院计算机网络信息中心);University of California, Los Angeles(加州大学洛杉矶分校);Peking University(北京大学)
Graph Your Way to Inspiration: Integrating Co-Author Graphs with Retrieval-Augmented Generation for Large Language Model Based Scientific Idea Generation
School of Computing Newcastle University(计算学院新castle大学)
MuISQA: Multi-Intent Retrieval-Augmented Generation for Scientific Question Answering
Zhongke Zidong Taichu (Beijing)(中科紫东太初(北京));Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学人工智能学院);Wuhan AI Research(武汉人工智能研究院)
MOT-SR: Multi-Objective Tool-Augmented Scientific Equation Discovery with Large Language Models
Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);University of the Chinese Academy of Sciences(中国科学院大学);School of Advanced Interdisciplinary Sciences(先进交叉科学学院);School of Astronomy and Space Science, University of the Chinese Academy of Sciences(中国科学院大学天文与空间科学学院)
HiPerRAG: High-Performance Retrieval Augmented Generation for Scientific Insights
Argonne National Laboratory(阿贡国家实验室);The University of Chicago(芝加哥大学);NVIDIA Inc.(NVIDIA公司);University of Illinois Chicago(伊利诺伊大学芝加哥分校);California Institute of Technology(加州理工学院)
Shanghai AI Laboratory(上海人工智能实验室);Harbin Institute of Technology(哈尔滨工业大学);Fudan University(复旦大学);CUHK(香港中文大学);Tsinghua University(清华大学);CUHK-Shenzhen(香港中文大学深圳校区);UESTC(电子科技大学);USTC(中国科学技术大学)
SpectraQuery: A Hybrid Retrieval-Augmented Conversational Assistant for Battery Science
Mechanical Engineering, Stanford University(斯坦福大学机械工程系);Applied Energy Division, SLAC National Accelerator Laboratory(SLAC国家加速器实验室应用能源部门);Institute of Computational and Mathematical Engineering, Stanford University(斯坦福大学计算与数学工程研究所)
Benchmarking Retrieval-Augmented Generation for Chemistry
Siebel School of Computing and Data Science, University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校Siebel计算与数据科学学院);National Library of Medicine, National Institutes of Health(美国国立卫生研究院国家医学图书馆)
Attention Expansion: Enhancing Keyphrase Extraction from Long Documents with Attention-Augmented Contextualized Embeddings
Institute for Research in Technology, ICAI School of Engineering, Comillas Pontifical University(技术研究所,ICAI工程学院,科米利亚斯宗座大学);DD-AIM, Senior Machine Learning Researcher(DD-AIM,高级机器学习研究员)
CoAuthorAI: A Human in the Loop System For Scientific Book Writing
Kexin Technology, Beijing(凯欣科技,北京);Institute for Sustainable Industries and Liveable Cities, Victoria University(可持续产业与宜居城市研究所,维多利亚大学);School of Information Technology, Deakin University(信息科技学院,德坎大学)
Multimodal Deep Learning for Scientific Imaging Interpretation
Science and Technology Division, Corning Incorporated(康宁公司科技事业部);Robert Frederick Smith School of Chemical and Biomolecular Engineering, Cornell University(康奈尔大学罗伯特·弗雷德里克·史密斯化学与生物分子工程学院)
本文提出多模态深度学习框架 GlassLLaVA,结合论文图文数据与 GPT-4,实现玻璃材料 SEM 图像的类人解释、特征识别和缺陷检测,并建立可对照研究答案的评估指标。
FlyAOC: Evaluating Agentic Ontology Curation of Drosophila Scientific Knowledge Bases
University of Michigan(密歇根大学);University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);University of California, Berkeley(加州大学伯克利分校);University of Washington(华盛顿大学)
ScIRGen: Synthesize Realistic and Large-Scale RAG Dataset for Scientific Research
The Hong Kong University of Science and Technology (Guangzhou)(香港科学与技术大学(广州));Institute of Tibetan Plateau Research, Chinese Academy of Sciences(中国科学院青藏高原研究所);Lanzhou University(兰州大学);College of Resources and Environment, University of Chinese Academy of Sciences(中国科学院大学资源与环境学院)