对齐大型语言模型以遵循指令并减少幻觉：通过有效的数据过滤
Aligning Large Language Models to Follow Instructions and Hallucinate Less via Effective Data Filtering
AI总结：
提出NOVA框架，通过内部一致性探测和语义等价识别过滤与模型知识一致的高质量指令数据，结合专家对齐奖励模型，减少幻觉并提升指令遵循能力。
摘要：
在指令微调阶段，使用包含模型不熟悉知识的数据训练LLM会诱发幻觉。为应对这一挑战，我们提出NOVA，一种新颖的框架，旨在识别与LLM已学知识高度一致的高质量数据，以减少幻觉。NOVA包含内部一致性探测（ICP）和语义等价识别（SEI），用于衡量LLM对指令数据的熟悉程度。具体而言，ICP通过计算多个自生成响应之间定制的一致性来评估LLM对给定指令的理解。SEI进一步通过将目标响应与生成响应进行比较，利用所提出的语义聚类和精心设计的投票策略，评估LLM对目标响应的熟悉度。最后，为确保所选样本的质量，我们引入一个专家对齐的奖励模型，考虑超越熟悉度的特征。通过考虑数据质量并避免不熟悉的数据，我们能够利用所选数据有效地对齐LLM，使其遵循指令并减少幻觉。
英文摘要：
Training LLMs on data containing unfamiliar knowledge during the instruction tuning stage can encourage hallucinations. To address this challenge, we introduce NOVA, a novel framework designed to identify high-quality data that aligns well with the LLM's learned knowledge to reduce hallucinations. NOVA includes Internal Consistency Probing (ICP) and Semantic Equivalence Identification (SEI) to measure how familiar the LLM is with instruction data. Specifically, ICP evaluates the LLM's understanding of the given instruction by calculating the tailored consistency among multiple self-generated responses. SEI further assesses the familiarity of the LLM with the target response by comparing it to the generated responses, using the proposed semantic clustering and well-designed voting strategy. Finally, to ensure the quality of selected samples, we introduce an expert-aligned reward model, considering characteristics beyond just familiarity. By considering data quality and avoiding unfamiliar data, we can utilize the selected data to effectively align LLMs to follow instructions and hallucinate less.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
