arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2405.05904cs.CL
收藏

在新知识上微调大语言模型会鼓励幻觉吗?

Does Fine-Tuning LLMs on New Knowledge Encourage Hallucinations?

Technion - Israel Institute of Technology(以色列理工学院);Google Research(谷歌研究院)

Zorik Gekhman, Gal Yona, Roee Aharoni, Matan Eyal, Amir Feder, Roi Reichart, Jonathan Herzig

AI总结:

本文通过闭卷问答受控实验发现,大语言模型难以通过微调获取新事实知识,且学会新知识会线性增加幻觉倾向,表明事实知识主要来自预训练,微调仅提升知识使用效率。

摘要:

当大语言模型通过监督微调进行对齐时,它们可能会遇到在预训练阶段未获得的新事实信息。人们通常推测,这可能会教会模型产生事实错误回答的幻觉行为,因为模型被训练生成并非基于其预训练已有知识的事实。在本工作中,我们研究了这种新知识暴露对微调后模型利用其已有知识能力的影响。为此,我们设计了一个受控实验设置,聚焦于闭卷问答,在其中改变引入新知识的微调样本比例。我们证明,大语言模型难以通过微调获取新的事实知识,因为引入新知识的微调样本比与模型已有知识一致的样本学习速度显著更慢。然而,我们还发现,随着这些包含新知识的样本最终被学会,它们会线性地增加模型产生幻觉的倾向。综合来看,我们的结果强调了通过微调引入新事实知识的风险,并支持以下观点:大语言模型主要通过预训练获取事实知识,而微调则教会它们更高效地使用这些知识。

英文摘要:

When large language models are aligned via supervised fine-tuning, they may encounter new factual information that was not acquired through pre-training. It is often conjectured that this can teach the model the behavior of hallucinating factually incorrect responses, as the model is trained to generate facts that are not grounded in its pre-existing knowledge. In this work, we study the impact of such exposure to new knowledge on the capability of the fine-tuned model to utilize its pre-existing knowledge. To this end, we design a controlled setup, focused on closed-book QA, where we vary the proportion of the fine-tuning examples that introduce new knowledge. We demonstrate that large language models struggle to acquire new factual knowledge through fine-tuning, as fine-tuning examples that introduce new knowledge are learned significantly slower than those consistent with the model's knowledge. However, we also find that as the examples with new knowledge are eventually learned, they linearly increase the model's tendency to hallucinate. Taken together, our results highlight the risk in introducing new factual knowledge through fine-tuning, and support the view that large language models mostly acquire factual knowledge through pre-training, whereas fine-tuning teaches them to use it more efficiently.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑