通过不确定性感知的指令微调平衡真实性与信息量
Balancing Truthfulness and Informativeness with Uncertainty-Aware Instruction Fine-Tuning
AI总结：
本文指出指令微调（IFT）中不熟悉知识会降低LLM真实性，提出$UNIT_{cut}$移除不熟悉知识以提升真实性，以及$UNIT_{ref}$让模型显式表达不确定性以维持信息量并减少幻觉。
摘要：
指令微调（IFT）能提升大型语言模型（LLM）的信息量，但可能降低其真实性。这种权衡的存在是因为IFT引导LLM生成预训练中未充分覆盖的长尾知识的回答。因此，模型变得更具信息量但在泛化到未见知识时准确性较低。在本文中，我们实证证明IFT数据集中的不熟悉知识会负面影响LLM的真实性，并提出两种新的IFT范式：$UNIT_{cut}$和$UNIT_{ref}$来解决此问题。$UNIT_{cut}$识别并移除IFT数据集中的不熟悉知识，以缓解其对模型真实性的影响，而$UNIT_{ref}$训练LLM识别自身的不确定性，并在回答末尾明确指示出来。实验表明，$UNIT_{cut}$显著提升了LLM的真实性，而$UNIT_{ref}$维持了高信息量，并通过区分确信与不确信的陈述减少了幻觉。
英文摘要：
Instruction fine-tuning (IFT) can increase the informativeness of large language models (LLMs), but may reduce their truthfulness. This trade-off arises because IFT steers LLMs to generate responses containing long-tail knowledge that was not well covered during pre-training. As a result, models become more informative but less accurate when generalizing to unseen tasks. In this paper, we empirically demonstrate how unfamiliar knowledge in IFT datasets can negatively affect the truthfulness of LLMs, and we introduce two new IFT paradigms, $UNIT_{cut}$ and $UNIT_{ref}$, to address this issue. $UNIT_{cut}$ identifies and removes unfamiliar knowledge from IFT datasets to mitigate its impact on model truthfulness, whereas $UNIT_{ref}$ trains LLMs to recognize their uncertainty and explicitly indicate it at the end of their responses. Our experiments show that $UNIT_{cut}$ substantially improves LLM truthfulness, while $UNIT_{ref}$ maintains high informativeness and reduces hallucinations by distinguishing between confident and uncertain statements.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
