arXivDaily arXiv每日学术速递 周一至周五更新
​通知:联邦节假日调休:哥伦布日 / 原住民日(Columbus Day / Indigenous Peoples' Day),今日(13日)无更新。
arXiv 2508.17225cs.CLcs.AI

SSFO:面向检索增强生成的自监督忠实性优化

SSFO: Self-Supervised Faithfulness Optimization for Retrieval-Augmented Generation

The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州));Chinese Academy of Sciences(中国科学院);Shanghai AI Lab(上海人工智能实验室);Tencent Hunyuan(腾讯文英)

Xiaqiang Tang, Yi Wang, Keyu Hu, Rui Xu, Chuang Li, Weigao Sun, Jian Li, Sihong Xie

AI总结:

针对RAG系统的忠实性幻觉问题,该研究提出首个自监督对齐方法SSFO,通过对比有无上下文的输出生成偏好对,结合改进的DPO损失实现无标注的忠实性优化,在多数据集上取得SOTA表现且泛化性强。

摘要:

检索增强生成(RAG)系统要求大语言模型(LLM)生成符合检索上下文的响应。然而,忠实性幻觉仍是关键挑战,现有方法往往需要高昂的监督成本与后训练过程,或是带来沉重的推理负担。为解决这些局限,我们提出自监督忠实性优化(SSFO),这是首个用于提升RAG忠实性的自监督对齐方法。SSFO通过对比模型在有上下文和无上下文条件下生成的输出,构建偏好数据对。该方法借助直接偏好优化(DPO)实现模型忠实性对齐,无需标注成本,也不会增加额外推理负担。我们从理论和实证两方面证明,SSFO利用了一种良性的“似然位移”机制,将概率质量从基于参数的词元转移到与上下文对齐的词元上。基于这一洞见,我们提出了一种改进的DPO损失函数以促进似然位移。综合评估表明,SSFO显著优于现有方法,在多个基于上下文的问答数据集上取得了最先进的忠实性表现。值得注意的是,SSFO展现出强泛化性,既能提升跨语言忠实性,又能保留通用的指令遵循能力。我们已在匿名链接https://github.com/chkwy/SSFO发布代码与模型。

英文摘要:

Retrieval-Augmented Generation (RAG) systems require Large Language Models (LLMs) to generate responses that are faithful to the retrieved context. However, faithfulness hallucination remains a critical challenge, as existing methods often require costly supervision and post-training or significant inference burdens. To overcome these limitations, we introduce Self-Supervised Faithfulness Optimization (SSFO), the first self-supervised alignment approach for enhancing RAG faithfulness. SSFO constructs preference data pairs by contrasting the model's outputs generated with and without the context. Leveraging Direct Preference Optimization (DPO), SSFO aligns model faithfulness without incurring labeling costs or additional inference burden. We theoretically and empirically demonstrate that SSFO leverages a benign form of \emph{likelihood displacement}, transferring probability mass from parametric-based tokens to context-aligned tokens. Based on this insight, we propose a modified DPO loss function to encourage likelihood displacement. Comprehensive evaluations show that SSFO significantly outperforms existing methods, achieving state-of-the-art faithfulness on multiple context-based question-answering datasets. Notably, SSFO exhibits strong generalization, improving cross-lingual faithfulness and preserving general instruction-following capabilities. We release our code and model at the anonymous link: https://github.com/chkwy/SSFO

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑