Context-DPO：面向上下文忠实性的语言模型对齐
Context-DPO: Aligning Language Models for Context-Faithfulness
AI总结：
本文提出首个增强LLMs上下文忠实性的对齐方法Context-DPO及评测基准ConFiQA，通过直接偏好优化显著提升了模型在RAG场景下的忠实度，且不损害其原有生成能力。
摘要：
大型语言模型（LLMs）的可靠响应需要遵循用户指令和检索到的信息。尽管对齐技术有助于LLMs与人类意图和价值观保持一致，但通过对齐来提升上下文忠实性仍未被充分探索。为此，我们提出Context-DPO，这是首个专门设计用于增强LLMs上下文忠实性的对齐方法。我们引入ConFiQA，这是一个模拟检索增强生成（RAG）场景并带有知识冲突的基准，用于评估上下文忠实性。通过利用ConFiQA中针对带有提供上下文问题的忠实且固执的回答，我们的Context-DPO通过直接偏好优化来对齐LLMs。大量实验表明，Context-DPO显著提升了上下文忠实性，在流行的开源模型上实现了35%至280%的提升。进一步分析表明，Context-DPO在保留LLMs生成能力的同时，为上下文利用提供了可解释的见解。我们的代码和数据已发布。
英文摘要：
Reliable responses from large language models (LLMs) require adherence to user instructions and retrieved information. While alignment techniques help LLMs align with human intentions and values, improving context-faithfulness through alignment remains underexplored. To address this, we propose $\textbf{Context-DPO}$, the first alignment method specifically designed to enhance LLMs' context-faithfulness. We introduce $\textbf{ConFiQA}$, a benchmark that simulates Retrieval-Augmented Generation (RAG) scenarios with knowledge conflicts to evaluate context-faithfulness. By leveraging faithful and stubborn responses to questions with provided context from ConFiQA, our Context-DPO aligns LLMs through direct preference optimization. Extensive experiments demonstrate that our Context-DPO significantly improves context-faithfulness, achieving 35% to 280% improvements on popular open-source models. Further analysis demonstrates that Context-DPO preserves LLMs' generative capabilities while providing interpretable insights into context utilization. Our code and data are released at https://github.com/byronBBL/Context-DPO
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
