利用 RLHF 在 LLM 中实现鲁棒的不可回答性识别与可信响应生成
Harnessing RLHF for Robust Unanswerability Recognition and Trustworthy Response Generation in LLMs
AI总结：
针对 CIR 中不可回答问题易引发幻觉的问题，本文提出 SALU，通过多任务学习与置信度引导的 RLHF 将检测和弃权（不执行）融入 LLM 生成过程，并在 C-IR_Answerability 上验证其准确率与可信度。
摘要：
会话式信息检索（CIR）系统虽然提供了直观的信息获取方式，却面临一个重大挑战：可靠地处理不可回答的问题，以防止生成误导性或幻觉内容。传统方法通常依赖外部分类器，这可能导致其与核心的生成式大语言模型（LLM）之间出现不一致。本文提出了 Self-Aware LLM for Unanswerability（SALU），这是一种将不可回答性检测深度直接集成到 LLM 生成过程中的新颖方法。SALU 使用多任务学习框架进行训练，同时面向标准问答（QA）以及针对不可回答查询的显式弃权（不执行）生成。关键在于，该方法纳入了一个由置信度分数引导的基于人类反馈的强化学习（RLHF）阶段，明确惩罚幻觉响应，并奖励恰当的弃权（不执行），从而培养对知识边界的内在自我意识。通过在自建 C-IR_Answerability 数据集上的大量实验，SALU 在正确回答问题或对问题弃权（不执行）的总体准确率方面，持续优于包括混合 LLM-分类器系统在内的强基线。人工评估进一步证实了 SALU 卓越的可靠性，其在事实性、恰当弃权（不执行）方面取得高分，最重要的是显著减少了幻觉，表明它具备鲁棒地“知道何时该说‘我不知道’”的能力。
英文摘要：
Conversational Information Retrieval (CIR) systems, while offering intuitive access to information, face a significant challenge: reliably handling unanswerable questions to prevent the generation of misleading or hallucinated content. Traditional approaches often rely on external classifiers, which can introduce inconsistencies with the core generative Large Language Models (LLMs). This paper introduces Self-Aware LLM for Unanswerability (SALU), a novel approach that deeply integrates unanswerability detection directly within the LLM's generative process. SALU is trained using a multi-task learning framework for both standard Question Answering (QA) and explicit abstention generation for unanswerable queries. Crucially, it incorporates a confidence-score-guided reinforcement learning with human feedback (RLHF) phase, which explicitly penalizes hallucinated responses and rewards appropriate abstentions, fostering intrinsic self-awareness of knowledge boundaries. Through extensive experiments on our custom-built C-IR_Answerability dataset, SALU consistently outperforms strong baselines, including hybrid LLM-classifier systems, in overall accuracy for correctly answering or abstaining from questions. Human evaluation further confirms SALU's superior reliability, achieving high scores in factuality, appropriate abstention, and, most importantly, a dramatic reduction in hallucination, demonstrating its ability to robustly "know when to say 'I don't know'."
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
