Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
Facebook AI Research（Facebook AI研究院）；University College London（伦敦大学学院）；New York University（纽约大学）
Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, Sebas… 展开作者
Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, Sebastian Riedel, Douwe Kiela
Large pre-trained language models have been shown to store factual knowledge in their parameters, and achieve state-of-the-art results when fine-tuned on downstream NLP tasks. However, their ability to access and precisely manipulate knowledge is still limited, and hence on knowledge-intensive tasks, their performance lags behind task-specific architectures. Additionally, providing provenance for their decisions and updating their world knowledge remain open research problems. Pre-trained models with a differentiable access mechanism to explicit non-parametric memory can overcome this issue, but have so far been only investigated for extractive downstream tasks. We explore a general-purpose fine-tuning recipe for retrieval-augmented generation (RAG) -- models which combine pre-trained parametric and non-parametric memory for language generation. We introduce RAG models where the parametric memory is a pre-trained seq2seq model and the non-parametric memory is a dense vector index of Wikipedia, accessed with a pre-trained neural retriever. We compare two RAG formulations, one which conditions on the same retrieved passages across the whole generated sequence, the other can use different passages per token. We fine-tune and evaluate our models on a wide range of knowledge-intensive NLP tasks and set the state-of-the-art on three open domain QA tasks, outperforming parametric seq2seq models and task-specific retrieve-and-extract architectures. For language generation tasks, we find that RAG models generate more specific, diverse and factual language than a state-of-the-art parametric-only seq2seq baseline.
DAMO-NLP at SemEval-2023 Task 2: A Unified Retrieval-augmented System for Multilingual Named Entity Recognition
DAMO Academy, Alibaba Group（阿里巴巴集团达摩院）；College of Computer Science and Technology, Zhejiang University（浙江大学计算机科学与技术学院）；School of Information Science and Technology, ShanghaiTech University（上海科技大学信息科学与技术学院）；Tsinghua Shenzhen International Graduate School, Tsinghua University（清华大学深圳国际研究生院）
DAMO-NLP at SemEval-2022 Task 11: A Knowledge-based System for Multilingual Named Entity Recognition
Zhejiang University（浙江大学）；Singapore University of Technology and Design（新加坡科技设计大学）；DAMO Academy, Alibaba Group（阿里巴巴集团达摩院）；ShanghaiTech University（上海科技大学）
A Systematic Review of Key Retrieval-Augmented Generation (RAG) Systems: Progress, Gaps, and Future Directions
Bredesen Center for Interdisciplinary Research, University of Tennessee, Knoxville, USA（跨学科研究中心，田纳西大学，肯塔基州 Knoxville）；National Center for Computational Sciences, Oak Ridge National Laboratory, Oak Ridge, USA（计算科学国家中心，橡树岭国家实验室，橡树岭）；University of Tennessee-Oak Ridge Innovation Institute, University of Tennessee, Knoxville, USA（田纳西大学橡树岭创新研究所，田纳西大学，肯塔基州 Knoxville）
On the Vulnerability of Applying Retrieval-Augmented Generation within Knowledge-Intensive Application Domains
Department of ECE University of Minnesota（电子工程系明尼苏达大学）；Data Science Institute University of Chicago（数据科学研究所芝加哥大学）；Carlson School of Management University of Minnesota（卡尔森管理学院明尼苏达大学）；Department of Surgery University of Minnesota（外科系明尼苏达大学）；Cisco Research（思科研究）；School of Statistics University of Minnesota（统计学学院明尼苏达大学）
Retrieval-Augmented Generation for Natural Language Processing: A Survey
City University of Hong Kong（香港城市大学）；Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）；McGill University, Mila（麦吉尔大学，MILA）；National Taiwan University（国立台湾大学）
Data Augmented Pipeline for Legal Information Extraction and Reasoning
Center for Juris-Informatics, ROIS-DS（司法信息中心，ROIS-DS）；Japan Advanced Institute of Science and Technology（日本先进科学研究所）；Research and Development Center for LLMs, NII（大语言模型研发中心，NII）
Learning to Deceive Knowledge Graph Augmented Models via Targeted Perturbation
Indian Institute of Technology, Delhi（印度理工学院德里分校）；University of Southern California（南加州大学）；Adobe Research（Adobe研究院）；Indian Institute of Technology, Kharagpur（印度理工学院卡拉格普尔分校）；Tsinghua University（清华大学）
GRAF: Graph Retrieval Augmented by Facts for Romanian Legal Multi-Choice Question Answering
National University of Science and Technology POLITEHNICA Bucharest, Faculty of Automatic Control and Computers（波兰技术大学布加勒斯特分校国家科学与技术大学，自动控制与计算机学院）；Technical University of Munich（慕尼黑技术大学）；Paris 1 Panthéon-Sorbonne University（巴黎第一大学）；University of Bucharest（布加勒斯特大学）
Leiden University Medical Centre（莱顿大学医学中心）；Lancaster University（兰卡斯特大学）；University of Manchester（曼彻斯特大学）；The Leiden Institute of Advanced Computer Science（莱顿先进计算机科学研究所）
CoCR-RAG: Enhancing Retrieval-Augmented Generation in Web Q&A via Concept-oriented Context Reconstruction
University of Southern Queensland（昆士兰大学）；University of Technology Sydney（新南威尔士大学）；The Hong Kong Polytechnic University（香港理工大学）；National University of Singapore（新加坡国立大学）；Qatar Computing Research Institute（卡塔尔计算研究所）；The Education University of Hong Kong（香港教育大学）
Adaptive Contrastive Decoding in Retrieval-Augmented Generation for Handling Noisy Contexts
Seoul National University（首尔大学）；Ewha Womans University（梨花女子大学）；NAVER Cloud（NAVER云）；KAIST AI（韩国科学技术院AI学院）；NAVER AI LAB（NAVER AI实验室）；Hanyang University（汉阳大学）
CausalRAG: Integrating Causal Graphs into Retrieval-Augmented Generation
Department of Computer and Data Sciences, Case Western Reserve University（计算机与数据科学系，凯斯西储大学）；Department of Design and Innovation, Case Western Reserve University（设计与创新系，凯斯西储大学）
The Web Is Your Oyster - Knowledge-Intensive NLP against a Very Large Web Corpus
Facebook AI Research（脸书人工智能研究院）；University College London（伦敦大学学院）；University of Mannheim（曼海姆大学）；ENS, PSL University（巴黎高等师范学院，巴黎文理研究大学）；Inria（法国国家信息与自动化研究所）；University of Washington（华盛顿大学）
Benchmarking Retrieval-Augmented Large Language Models in Biomedical NLP: Application, Robustness, and Self-Awareness
Division of Computational Health Sciences, Department of Surgery University of Minnesota（大学医学计算科学系，外科系）；Institute for Health Informatics University of Minnesota（健康信息学研究所，明尼苏达大学）；Department of Electrical and Computer Engineering University of Minnesota（电气与计算机工程系，明尼苏达大学）
LLM-Augmented Symptom Analysis for Cardiovascular Disease Risk Prediction: A Clinical NLP
University of Houston（休斯顿大学）；Columbia University（哥伦比亚大学）；University of Michigan（密歇根大学）；Carnegie Mellon University（卡内基梅隆大学）；University of California, Riverside（加州大学河滨分校）
QU-NLP at CheckThat! 2025: Multilingual Subjectivity in News Articles Detection using Feature-Augmented Transformer Models with Sequential Cross-Lingual Fine-Tuning
