KnowShiftQA: How Robust are RAG Systems when Textbook Knowledge Shifts in K-12 Education?
Department of Computer Science and Engineering, HKUST, Hong Kong SAR, China（计算机科学与工程系，香港科技大学，香港特别行政区，中国）；Department of Mechanical Engineering, The University of Tokyo, Tokyo, Japan（机械工程系，东京大学，东京，日本）
Tianshi Zheng, Weihan Li, Jiaxin Bai, Weiqi Wang, Yangqiu Song
Retrieval-Augmented Generation (RAG) systems show remarkable potential as question answering tools in the K-12 Education domain, where knowledge is typically queried within the restricted scope of authoritative textbooks. However, discrepancies between these textbooks and the parametric knowledge inherent in Large Language Models (LLMs) can undermine the effectiveness of RAG systems. To systematically investigate RAG system robustness against such knowledge discrepancies, we introduce KnowShiftQA. This novel question answering dataset simulates these discrepancies by applying deliberate hypothetical knowledge updates to both answers and source documents, reflecting how textbook knowledge can shift. KnowShiftQA comprises 3,005 questions across five subjects, designed with a comprehensive question typology focusing on context utilization and knowledge integration. Our extensive experiments on retrieval and question answering performance reveal that most RAG systems suffer a substantial performance drop when faced with these knowledge discrepancies. Furthermore, questions requiring the integration of contextual (textbook) knowledge with parametric (LLM) knowledge pose a significant challenge to current LLMs.
Enhancing textual textbook question answering with large language models and retrieval augmented generation
Faculty of Computing and Information Technology, King Abdulaziz University（阿卜杜勒阿齐兹国王大学计算与信息技术学院）；College of Computer and Information Science, Imam Mohammad Ibn Saud Islamic University (IMSIU)（伊玛目穆罕默德·本·沙特伊斯兰大学计算机与信息科学学院）；School of Computing, Macquarie University（麦考瑞大学计算学院）
Benchmarking Retrieval-Augmented Generation for Chemistry
Siebel School of Computing and Data Science, University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校Siebel计算与数据科学学院）；National Library of Medicine, National Institutes of Health（美国国立卫生研究院国家医学图书馆）
Evaluating Multimodal Large Language Models on Educational Textbook Question Answering
Faculty of Computing and Information Technology（计算与信息科技学院）；King Abdulaziz University（阿卜杜勒阿齐兹大学）；FAST School of Computing（快速计算学院）；National University of Computer and Emerging Sciences（国家计算机与新兴科学大学）；School of Computing（计算学院）；Macquarie University（麦考瑞大学）；Center of Research Excellence in AI and Data Science（人工智能与数据科学卓越研究中心）
TeachMateGPT: A Multi-Agent Knowledge-Grounded Framework for Pedagogical Assessment Generation from Science Curriculum Materials
Ahsanullah University of Science and Technology（阿赫桑乌拉科技大学）；American International University - Bangladesh（孟加拉国美国国际大学）；Jashore University of Science and Technology（杰索尔科技大学）
Allen Institute for AI（艾伦人工智能研究所）；University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；University of Southern California（南加州大学）；University of Washington（华盛顿大学）
本文提出网络规模高质量数据存储 CompactDS，结合内存 ANN 检索与磁盘精确搜索，使最小化 RAG 在多个推理密集型基准上一致提升准确率，并可与搜索引擎及复杂智能体 RAG 系统相媲美。
A Systematic Study of Retrieval Pipeline Design for Retrieval-Augmented Medical Question Answering
Department of Mechatronics & Industrial Engineering, Chittagong University of Engineering & Technology（吉大港工程与技术大学机电与工业工程系）；Department of Mechanical Engineering, Chittagong University of Engineering & Technology（吉大港工程与技术大学机械工程系）
OralAgent: Integrating Reasoning, Tools, and Knowledge for Interactive Dental Image Analysis
Faculty of Dentistry, the University of Hongkong, Hong Kong SAR, China（香港大学牙科学院，中国香港特别行政区）；Department of Electrical and Computer Engineering, University of Pittsburgh, Pittsburgh, PA, USA（匹兹堡大学电气与计算机工程系，美国宾夕法尼亚州匹兹堡）；Shenzhen University, China（深圳大学，中国）；Department of Craniomaxillofacial Surgery, Shanghai Ninth People’s Hospital, China（上海第九人民医院口腔颌面外科部，中国）；Nanyang technological University, Singapore（南洋理工大学，新加坡）；School of Biomedical Engineering, Southern Medical University, China（南方医科大学生物医学工程学院，中国）；Singapore University of Technology and Design, Singapore（新加坡科技设计大学，新加坡）；University of Auckland, new zealand（奥克兰大学，新西兰）；Shanghai Artificial Intelligence Laboratory , China（上海人工智能实验室，中国）
MedTutor: A Retrieval-Augmented LLM System for Case-Based Medical Education
Department of Computer Science, Yale University（耶鲁大学计算机科学系）；Department of Radiology and Biomedical Imaging, Yale School of Medicine（耶鲁医学院放射学与生物医学成像系）；Interdisciplinary Program for Bioengineering, Seoul National University（首尔国立大学生物工程跨学科项目）
