arXiv 2502.059112025-02-09cs.CL

GRAIT：梯度驱动的拒绝感知指令调优用于有效缓解幻觉

GRAIT: Gradient-Driven Refusal-Aware Instruction Tuning for Effective Hallucination Mitigation

Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）；Peking University（北京大学）；Southwest Jiaotong University（西南交通大学）；Shanghai Jiaotong University（上海交通大学）；Chinese University of Hong Kong（香港中文大学）

Runchuan Zhu, Zinco Jiang, Jiang Wu, Zhipeng Ma, Jiahe Song, Fengshuo Bai, Dahua Lin, Lijun Wu, Conghui He

备注：: Equal contribution: Runchuan Zhu, Zinco Jiang, Jiang Wu; Corresponding author: Conghui He

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

针对拒绝感知指令调优（RAIT）的两大挑战，本文从梯度视角提出GRAIT框架，通过梯度驱动样本选择和自适应加权机制平衡拒绝准确性与有用性，在问答任务中显著优于现有方法，有效缓解幻觉。

摘要：

拒绝感知指令调优（RAIT）旨在通过提升大语言模型（LLMs）拒绝超出其知识范围问题的能力来增强模型，从而减少幻觉并提高可靠性。有效的RAIT必须解决两个关键挑战：一是有效拒绝未知问题以最小化幻觉；二是避免过度拒绝，确保可正确回答的问题不被拒绝，从而维持LLM输出的有用性。本文中，我们从基于梯度的视角得出深刻观察，提出梯度驱动的拒绝感知指令调优框架GRAIT来解决这两个挑战：（1）采用梯度驱动的样本选择以有效减少幻觉；（2）在微调期间引入自适应加权机制以降低过度拒绝风险，实现准确拒绝与维持有用响应的平衡。在开放式和多项选择问答任务上的实验评估表明，GRAIT在整体性能上显著优于现有RAIT方法。源代码和数据将在https://github.com/opendatalab/GRAIT 上公开。

英文摘要：

Refusal-Aware Instruction Tuning (RAIT) aims to enhance Large Language Models (LLMs) by improving their ability to refuse responses to questions beyond their knowledge, thereby reducing hallucinations and improving reliability. Effective RAIT must address two key challenges: firstly, effectively reject unknown questions to minimize hallucinations; secondly, avoid over-refusal to ensure questions that can be correctly answered are not rejected, thereby maintain the helpfulness of LLM outputs. In this paper, we address the two challenges by deriving insightful observations from the gradient-based perspective, and proposing the Gradient-driven Refusal Aware Instruction Tuning Framework GRAIT: (1) employs gradient-driven sample selection to effectively minimize hallucinations and (2) introduces an adaptive weighting mechanism during fine-tuning to reduce the risk of over-refusal, achieving the balance between accurate refusals and maintaining useful responses. Experimental evaluations on open-ended and multiple-choice question answering tasks demonstrate that GRAIT significantly outperforms existing RAIT methods in the overall performance. The source code and data will be available at https://github.com/opendatalab/GRAIT .

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。