从LLM生成的反馈中学习摘要
Learning to Summarize from LLM-generated Feedback
AI总结：
针对LLM生成摘要的幻觉和遗漏等问题，研究构建了多维度反馈数据集FeedSum，通过监督微调和直接偏好优化方法训练出SummLlama3-8b模型，在生成人类偏好摘要上超越近10倍大的Llama3-70b-instruct。
摘要：
开发有效的文本摘要器仍然是一项挑战，因为LLM生成的摘要存在幻觉、关键信息遗漏和冗长等问题。本研究探索利用LLM生成的反馈来提高摘要质量，使摘要符合人类对忠实性、完整性和简洁性的偏好。我们引入了FeedSum，这是一个大规模数据集，包含针对不同领域不同质量摘要的多维度LLM反馈。实验表明，反馈的质量、维度和粒度如何影响偏好学习，并揭示高质量、多维度、细粒度的反馈能显著改善摘要生成。我们还比较了使用此反馈的两种方法：监督微调和直接偏好优化。最后，我们引入了SummLlama3-8b模型，该模型在生成人类偏好的摘要方面优于近10倍大的Llama3-70b-instruct，证明较小模型在适当训练下也能实现更优性能。完整数据集和SummLlama3-8B模型可在 https://huggingface.co/datasets/DISLab/FeedSum 和 https://huggingface.co/DISLab/SummLlama3-8B 获取。
英文摘要：
Developing effective text summarizers remains a challenge due to issues like hallucinations, key information omissions, and verbosity in LLM-generated summaries. This work explores using LLM-generated feedback to improve summary quality by aligning the summaries with human preferences for faithfulness, completeness, and conciseness. We introduce FeedSum, a large-scale dataset containing multi-dimensional LLM feedback on summaries of varying quality across diverse domains. Our experiments show how feedback quality, dimensionality, and granularity influence preference learning, revealing that high-quality, multi-dimensional, fine-grained feedback significantly improves summary generation. We also compare two methods for using this feedback: supervised fine-tuning and direct preference optimization. Finally, we introduce SummLlama3-8b, a model that outperforms the nearly 10x larger Llama3-70b-instruct in generating human-preferred summaries, demonstrating that smaller models can achieve superior performance with appropriate training. The full dataset and SummLlama3-8B model are available at https://huggingface.co/datasets/DISLab/FeedSum and https://huggingface.co/DISLab/SummLlama3-8B.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
