通过来自多个不完美指标的偏好学习优化摘要的事实一致性
Optimising Factual Consistency in Summarisation via Preference Learning from Multiple Imperfect Metrics
AI总结：
提出一种自动化训练流程，通过聚合多个弱事实性指标的分数并映射为偏好，过滤高分歧样本，利用词汇相似摘要对进行偏好学习，从而提升摘要的事实一致性。
摘要：
使用评估指标作为奖励的强化学习被广泛用于增强语言模型的特定能力。然而，对于事实一致性摘要等任务，现有指标仍不完善，限制了其作为塑造模型行为的信号的有效性。虽然单个事实性指标不可靠，但它们的组合可以更有效地捕捉多样的事实错误。我们利用这一见解，引入了一种自动化训练流程，通过聚合来自不同弱指标的分数来提高摘要的事实一致性。我们的方法通过将分数映射到偏好并过滤掉指标之间高度不一致的情况，避免了复杂的奖励塑造。对于每个源文档，我们通过改变解码策略生成词汇相似的摘要对，使模型能够从由细微词汇差异引起的事实差异中学习。这种方法仅使用源文档构建高质量偏好数据集。实验表明，从早期的编码器-解码器架构到现代大型语言模型，模型均获得一致的事实性提升，较小的模型能达到与较大模型相当的事实性。
英文摘要：
Reinforcement learning with evaluation metrics as rewards is widely used to enhance specific capabilities of language models. However, for tasks such as factually consistent summarisation, existing metrics remain underdeveloped, limiting their effectiveness as signals for shaping model behaviour.While individual factuality metrics are unreliable, their combination can more effectively capture diverse factual errors. We leverage this insight to introduce an automated training pipeline that improves factual consistency in summaries by aggregating scores from different weak metrics. Our approach avoids the need for complex reward shaping by mapping scores to preferences and filtering out cases with high disagreement between metrics. For each source document, we generate lexically similar summary pairs by varying decoding strategies, enabling the model to learn from factual differences caused by subtle lexical differences. This approach constructs a high-quality preference dataset using only source documents.Experiments demonstrate consistent factuality gains across models, ranging from early encoder-decoder architectures to modern large language models, with smaller models reaching comparable factuality to larger ones.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
