事实性微调的奇特案例：模型的内部信念能改善事实性
The Curious Case of Factuality Finetuning: Models' Internal Beliefs Can Improve Factuality
AI总结：
研究微调数据事实性与长文本生成幻觉的关系，发现基于模型内部信念过滤的模型生成数据微调比金标准数据更能提升跨领域事实性。
摘要：
语言模型容易出现幻觉——生成事实不正确的文本。在高质量事实信息上微调模型可能减少幻觉，但担忧依然存在：获取事实金标准数据可能很昂贵，且在正确但不熟悉的数据上训练可能导致下游出现更多幻觉。从业者应该在什么数据上微调以缓解语言模型中的幻觉？在这项工作中，我们研究了微调数据的事实性与长文本生成任务中幻觉普遍性之间的关系。出乎意料的是，我们发现在事实金标准数据上微调不如在模型认为事实正确的模型生成数据上微调有帮助。接下来，我们评估了应用于事实金标准数据和模型生成数据的过滤策略，发现基于模型自身内部判断过滤的模型生成数据进行微调，通常比其他配置带来更好的整体事实性：在按模型判断过滤的金标准数据上训练、仅在金标准数据上训练，或在由金标准数据支持的模型生成数据上训练。这些事实性改进在我们研究的三个领域中均能迁移，表明模型自身的信念可为事实性提供强大信号。
英文摘要：
Language models are prone to hallucination - generating text that is factually incorrect. Finetuning models on high-quality factual information can potentially reduce hallucination, but concerns remain; obtaining factual gold data can be expensive and training on correct but unfamiliar data may potentially lead to even more downstream hallucination. What data should practitioners finetune on to mitigate hallucinations in language models? In this work, we study the relationship between the factuality of finetuning data and the prevalence of hallucinations in long-form generation tasks. Counterintuitively, we find that finetuning on factual gold data is not as helpful as finetuning on model-generated data that models believe to be factual. Next, we evaluate filtering strategies applied on both factual gold data and model-generated data, and find that finetuning on model-generated data that is filtered by models' own internal judgments often leads to better overall factuality compared to other configurations: training on gold data filtered by models' judgments, training on gold data alone, or training on model-generated data that is supported by gold data. These factuality improvements transfer across three domains we study, suggesting that a models' own beliefs can provide a powerful signal for factuality.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
