语言模型（LM）会记忆大量事实性知识，并在多种任务和领域中表现出强劲性能。然而，人们观察到，在处理不太流行或低频的概念与实体时（例如在领域特定应用中），其性能会下降。提升 LM 在低频主题上性能的两种突出方法是：检索增强生成（RAG）和基于合成数据的微调（FT）。本文探索并评估了 RAG 与 FT 在定制 LM 以处理问答任务中低频实体方面的影响。我们对 12 个不同规模和类型的 LM，以及不同的微调、数据增强和检索模型进行了大量实验。研究结果表明，尽管 FT 提升了模型在不同流行度实体上的性能，但 RAG 大幅超越 FT，尤其是在最不流行的事实性知识方面。此外，改进检索和数据增强技术会放大 RAG 与 FT 两种方法的成功效果。微调虽然有益于小型 LM，但需要大量资源。为解决这一问题，我们提出了新的 Stimulus RAG 方法，其效果超越了基于微调的方法，从而无需为丰富 LM 的冷门事实性知识而执行代价高昂的数据增强和微调步骤。代码可在 https://github.com/informagi/RAGvsFT 获取。
英文摘要：
Language Models (LMs) memorize a vast amount of factual knowledge, exhibiting strong performance across diverse tasks and domains. However, it has been observed that the performance diminishes when dealing with less-popular or low-frequency concepts and entities, for example in domain specific applications. The two prominent approaches to enhance the performance of LMs on low-frequent topics are: Retrieval Augmented Generation (RAG) and fine-tuning (FT) over synthetic data. This paper explores and evaluates the impact of RAG and FT on customizing LMs in handling low-frequency entities on question answering tasks. We conduct extensive experiments on twelve LMs of varying size and type and different fine tuning, data augmentation, and retrieval models. Our findings indicate that while FT boosts the performance across entities of varying popularity, RAG surpasses FT by a large margin particularly for least popular factual knowledge. Additionally, the success of both RAG and FT approaches is amplified by improving retrieval and data augmentation techniques. Fine tuning, while beneficial for small LMs, requires extensive resources. To address this issue, we propose the new Stimulus RAG approach that surpasses the effectiveness of fine tuning based approaches, thereby eliminating the need for the costly data augmentation and fine tuning step for enriching LMs with less popular factual knowledge. The code is available at \url{https://github.com/informagi/RAGvsFT}.
Enhancing Retrieval-Augmented LMs with a Two-stage Consistency Learning Compressor
College of Intelligence and Computing, Tianjin University（天津大学智能与计算学部）；Artificial Intelligence Laboratory, China Mobile Communication Group Tianjin Co., Ltd.（中国移动通信集团天津有限公司人工智能实验室）；University College London（伦敦大学学院）
DioR: Adaptive Cognitive Detection and Contextual Retrieval Optimization for Dynamic Retrieval-Augmented Generation
Zhejiang Key Laboratory of Intelligent Education Technology and Application, Zhejiang Normal University（浙江智能教育技术与应用重点实验室，浙江师范大学）；School of Computer Science and Technology, Zhejiang Normal University（浙江师范大学计算机科学与技术学院）；School of Computer Science and Engineering, Southeast University（东南大学计算机科学与工程学院）；Department of Computer Science and Engineering, Hong Kong University of Science and Technology（香港科技大学计算机科学与工程系）；School of Computer Science and Technology, Soochow University（苏州大学计算机科学与技术学院）
RAR$^2$: Retrieval-Augmented Medical Reasoning via Thought-Driven Retrieval
Department of Computing, The Hong Kong Polytechnic University（计算机系，香港理工大学）；Research Institute of Trustworthy Autonomous Systems and Department of Computer Science and Engineering, Southern University of Science and Technology（可信自主系统研究院和计算机科学与工程系，南方科技大学）
ParetoRAG: Leveraging Sentence-Context Attention for Robust and Efficient Retrieval-Augmented Generation
Institute of Information Engineering, Chinese Academy of Sciences（中国科学院信息工程研究所）；School of Cybersecurity, University of Chinese Academy of Sciences（中国科学院大学网络空间安全学院）；Alibaba Group（阿里巴巴集团）
RPO: Retrieval Preference Optimization for Robust Retrieval-Augmented Generation
National Engineering Research Center of Speech and Language Information Processing, University of Science and Technology of China（语音与语言信息处理国家工程研究中心，中国科学技术大学）；State Key Laboratory of Cognitive Intelligence, iFLYTEK Research（认知智能国家重点实验室，iFLYTEK研究院）
GraphSearch: An Agentic Deep Searching Workflow for Graph Retrieval-Augmented Generation
IDEA Research, International Digital Economy Academy（IDEA研究机构、国际数字经济学院）；Hong Kong University of Science and Technology (Guangzhou)（香港科学与技术大学（广州））；DataArc Tech Ltd.（DataArc科技有限公司）；Hithink RoyalFlush Information Network Co., Ltd（Hithink皇家Flush信息网络有限公司）
LevelRAG: Enhancing Retrieval-Augmented Generation with Multi-hop Logic Planning over Rewriting Augmented Searchers
Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）；Chinese Academy of Sciences（中国科学院）；University of Chinese Academy of Sciences（中国科学院大学）；Harbin Institute of Technology (Shenzhen)（哈尔滨工业大学（深圳））
A Large Language Model-based Framework for Semi-Structured Tender Document Retrieval-Augmented Generation
School of Information Management Sun Yat-sen University Guangzhou, China；School of Information Management Sun Yat-sen University Guangzhou, China corresponding author
A Systematic Study of Retrieval Pipeline Design for Retrieval-Augmented Medical Question Answering
Department of Mechatronics & Industrial Engineering, Chittagong University of Engineering & Technology（吉大港工程与技术大学机电与工业工程系）；Department of Mechanical Engineering, Chittagong University of Engineering & Technology（吉大港工程与技术大学机械工程系）
CausalRAG: Integrating Causal Graphs into Retrieval-Augmented Generation
Department of Computer and Data Sciences, Case Western Reserve University（计算机与数据科学系，凯斯西储大学）；Department of Design and Innovation, Case Western Reserve University（设计与创新系，凯斯西储大学）
