利用两阶段一致性学习压缩器增强检索增强语言模型
Enhancing Retrieval-Augmented LMs with a Two-stage Consistency Learning Compressor
AI总结：
本文提出一种用于检索增强语言模型的两阶段一致性学习压缩方法，通过结合对比与一致性学习生成对齐教师模型且忠实于原文的摘要，显著提升了问答任务的精确度与效率。
摘要：
尽管检索增强语言模型（RALMs）已被广泛使用，但将这些模型与检索机制无缝集成以提升基于文档任务的性能仍然具有挑战性。虽然一些检索后处理的检索增强生成（RAG）方法取得了成功，但大多数仍缺乏区分相关信息与无关信息的能力，导致生成的输出存在潜在的不一致性并降低了精确度，进而影响语言模型响应的真实性。为了解决这些局限性，本文提出了一种新颖的两阶段一致性学习方法，用于检索增强语言模型中的检索信息压缩以提升性能。通过引入一致性学习，旨在生成保持连贯性并与教师模型预期语义表示对齐的摘要，同时提高对原始检索文档的忠实度。所提出的方法在多个数据集上进行了实证验证，证明了在问答任务的精确度和效率上有显著提升。它优于现有基线，并展示了在检索增强生成框架中结合对比学习和一致性学习范式的协同效应。
英文摘要：
Despite the prevalence of retrieval-augmented language models (RALMs), the seamless integration of these models with retrieval mechanisms to enhance performance in document-based tasks remains challenging. While some post-retrieval processing Retrieval-Augmented Generation (RAG) methods have achieved success, most still lack the ability to distinguish pertinent from extraneous information, leading to potential inconsistencies and reduced precision in the generated output, which subsequently affects the truthfulness of the language model's responses. To address these limitations, this work proposes a novel two-stage consistency learning approach for retrieved information compression in retrieval-augmented language models to enhance performance. By incorporating consistency learning, the aim is to generate summaries that maintain coherence and alignment with the intended semantic representations of a teacher model while improving faithfulness to the original retrieved documents. The proposed method is empirically validated across multiple datasets, demonstrating notable enhancements in precision and efficiency for question-answering tasks. It outperforms existing baselines and showcases the synergistic effects of combining contrastive and consistency learning paradigms within the retrieval-augmented generation framework.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
