Auto-GDA：面向检索增强生成中高效事实验证的自动领域适应
Auto-GDA: Automatic Domain Adaptation for Efficient Grounding Verification in Retrieval-Augmented Generation
AI总结：
针对真实 RAG 场景中轻量 NLI 验证模型领域不匹配且缺少标注的问题，本文提出 Auto-GDA，利用教师弱标签与离散优化迭代生成并筛选合成数据，实现无监督领域适应。
摘要：
尽管检索增强生成（RAG）已被证明能够提升大型语言模型（LLM）输出的事实性，但 LLM 仍会受到幻觉影响，生成不正确或不相关的信息。一种常见的检测策略是再次提示 LLM，以评估其回答是否基于检索到的证据，但这种方法成本高昂。另一种方案是在推理时使用轻量级自然语言推理（NLI）模型进行高效的事实验证。尽管现有的预训练 NLI 模型提供了潜在解决方案，但在真实的 RAG 输入上，其性能仍不及更大规模模型。RAG 输入比用于训练 NLI 模型的大多数数据集更复杂，并且具有底层知识库特有的特征，因此需要使 NLI 模型适应特定目标领域。此外，目标领域缺少标注实例，使得通过微调等监督式领域适应方法不可行。为应对这些挑战，我们提出了 Automatic Generative Domain Adaptation（Auto-GDA，自动生成式领域适应）。该框架通过合成数据生成实现无监督领域适应。不同于以往依赖人工设计的过滤和增强策略的方法，Auto-GDA 采用迭代流程，利用效率较低的教师模型给出的弱标签，并通过离散优化选择最有潜力的增强样本，从而持续提升生成样本的质量。实验结果证明了本方法的有效性：使用 Auto-GDA 在合成数据上微调的模型通常会超过教师模型的性能，并以 LLM 10% 的计算成本达到与其相当的性能水平。
英文摘要：
While retrieval-augmented generation (RAG) has been shown to enhance factuality of large language model (LLM) outputs, LLMs still suffer from hallucination, generating incorrect or irrelevant information. A common detection strategy involves prompting the LLM again to assess whether its response is grounded in the retrieved evidence, but this approach is costly. Alternatively, lightweight natural language inference (NLI) models for efficient grounding verification can be used at inference time. While existing pre-trained NLI models offer potential solutions, their performance remains subpar compared to larger models on realistic RAG inputs. RAG inputs are more complex than most datasets used for training NLI models and have characteristics specific to the underlying knowledge base, requiring adaptation of the NLI models to a specific target domain. Additionally, the lack of labeled instances in the target domain makes supervised domain adaptation, e.g., through fine-tuning, infeasible. To address these challenges, we introduce Automatic Generative Domain Adaptation (Auto-GDA). Our framework enables unsupervised domain adaptation through synthetic data generation. Unlike previous methods that rely on handcrafted filtering and augmentation strategies, Auto-GDA employs an iterative process to continuously improve the quality of generated samples using weak labels from less efficient teacher models and discrete optimization to select the most promising augmented samples. Experimental results demonstrate the effectiveness of our approach, with models fine-tuned on synthetic data using Auto-GDA often surpassing the performance of the teacher model and reaching the performance level of LLMs at 10% of their computational cost.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
