RAGognizer: 通过检测头整合实现hallucination-aware微调
RAGognizer: Hallucination-Aware Fine-Tuning via Detection Head Integration
AI总结：
RAGognizer通过整合轻量级检测头，实现hallucination-aware微调，提升内部状态分离性并减少生成hallucination。
摘要：
Retrieval-Augmented Generation (RAG) 是广泛用于通过外部信息增强大型语言模型 (LLMs) 输入的范式，如最近或领域特定的知识。然而，当前模型仍会产生封闭领域hallucination，并生成未被检索上下文支持的内容。当前检测方法通常将hallucination视为事后问题，依赖于黑箱一致性检查或冻结内部表示上的探测。在本工作中，我们证明基于内部状态表示的hallucination检测也可以作为直接训练信号。我们引入了包含自然发生的封闭领域hallucination的RAGognize数据集以及RAGognizer，一种整合轻量级检测头到LLM中的hallucination-aware微调方法，允许语言建模和hallucination检测的联合优化。这种联合目标迫使模型在提高内部状态分离性的同时学习生成良好结构和有意义的响应。在多个基准测试中，RAGognizer在token级hallucination检测上达到最先进的水平，同时在生成过程中显著减少hallucination率，而不影响语言质量和相关性。
英文摘要：
Retrieval-Augmented Generation (RAG) is widely used to augment the input to Large Language Models (LLMs) with external information, such as recent or domain-specific knowledge. Nonetheless, current models still produce closed-domain hallucinations and generate content that is unsupported by the retrieved context. Current detection approaches typically treat hallucination as a post-hoc problem, relying on black-box consistency checks or probes over frozen internal representations. In this work, we demonstrate that hallucination detection based on internal state representation can also serve as a direct training signal. We introduce RAGognize, a dataset of naturally occurring closed-domain hallucinations with token-level annotations, and RAGognizer, a hallucination-aware fine-tuning approach that integrates a lightweight detection head into an LLM, allowing for the joint optimization of language modeling and hallucination detection. This joint objective forces the model to improve the separability of its internal states regarding hallucinations while simultaneously learning to generate well-formed and meaningful responses. Across multiple benchmarks, RAGognizer achieves state-of-the-art token-level hallucination detection while substantially reducing hallucination rates during generation, without degrading language quality or relevance.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
