虚拟合成数据可通过先决条件学习提升 LLM 的事实性
Fictitious Synthetic Data Can Improve LLM Factuality via Prerequisite Learning
AI总结：
本文提出 Prereq-Tune，通过先决条件学习解耦知识与技能，并结合虚拟合成数据缓解预训练—微调知识不一致，从而提升 LLM 在问答和长文本生成中的事实性。
摘要：
近期研究指出，LLM 幻觉的一个加剧因素是预训练与微调之间的知识不一致：不熟悉的微调数据会误导 LLM 编造看似合理但错误的输出。本文提出一种名为 Prereq-Tune 的新型微调策略，以解决这种知识不一致并减少幻觉。从根本上说，Prereq-Tune 将技能学习与知识学习解耦，使模型只学习任务技能，而不受知识不一致影响。为此，Prereq-Tune 引入额外的先决条件学习阶段，用于学习 SFT 所需的必要知识，从而让后续 SFT 仅关注任务技能。Prereq-Tune 还可与虚拟合成数据结合，增强 LLM 输出对其内部知识的扎根。实验表明，在短问答和长文本生成任务中，Prereq-Tune 在提升 LLM 事实性方面优于现有基线。它还为 LLM 的知识受控生成开辟了新的可能性。代码位于 https://github.com/UCSB-NLP-Chang/Prereq_tune.git。
英文摘要：
Recent studies have identified one aggravating factor of LLM hallucinations as the knowledge inconsistency between pre-training and fine-tuning, where unfamiliar fine-tuning data mislead the LLM to fabricate plausible but wrong outputs. In this paper, we propose a novel fine-tuning strategy called Prereq-Tune to address this knowledge inconsistency and reduce hallucinations. Fundamentally, Prereq-Tune disentangles the learning of skills and knowledge, so the model learns only the task skills without being impacted by the knowledge inconsistency. To achieve this, Prereq-Tune introduces an additional prerequisite learning stage to learn the necessary knowledge for SFT, allowing subsequent SFT to focus only on task skills. Prereq-Tune can also be combined with fictitious synthetic data to enhance the grounding of LLM outputs to their internal knowledge. Experiments show that Prereq-Tune outperforms existing baselines in improving LLM's factuality across short QA and long-form generation tasks. It also opens new possibilities for knowledge-controlled generation in LLMs. Our code is available at https://github.com/UCSB-NLP-Chang/Prereq_tune.git.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
