arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2411.05232cs.CLcs.AI
收藏

Abstract2Appendix：学术评审增强LLM长上下文能力

Abstract2Appendix: Academic Reviews Enhance LLM Long-Context Capabilities

TIFIN；Clark University（克拉克大学）；Shopee；University of Massachusetts Boston（马萨诸塞大学波士顿分校）

Shengzhi Li, Kittipat Kampa, Rongyu Lin, Bohang Li, Shichao Pei

备注：
We share our latest dataset on https://github.com/findalexli/Abstract2Appendix
阅读 PDF
arXiv 页面 ↗HTML 原文 ↗

AI总结：

本研究利用学术评审数据微调LLM，通过DPO方法在Qasper基准上提升性能，证明高质量人类评审在长上下文任务中的优势。

摘要：

大型语言模型（LLM）在各种任务中表现出色，但其处理长上下文阅读的能力仍具挑战性。本研究探讨了利用高质量学术同行评审数据对LLM进行微调以增强其长上下文能力的有效性。我们比较了直接偏好优化（DPO）方法与监督微调（SFT）方法，证明了DPO的优越性和数据效率。实验表明，仅使用2000个样本，微调模型在Qasper基准上比phi-3提升了4.04分，并实现了2.6%的增幅。尽管面临数据规模和处理成本方面的限制，本研究强调了DPO和高质量数据在提升LLM性能方面的潜力。此外，零样本基准结果表明，聚合的高质量人类评审 overwhelmingly 优于LLM生成的响应，即使对于像GPT-4o这样最强大的模型也是如此。这表明高质量人类评审在信息丰富度、推理能力和长上下文检索方面极具价值，这些能力即便是最先进的模型也尚未完全掌握。这些发现凸显了利用人类评审进一步推动该领域发展的巨大效用。

英文摘要：

Large language models (LLMs) have shown remarkable performance across various tasks, yet their ability to handle long-context reading remains challenging. This study explores the effectiveness of leveraging high-quality academic peer review data for fine-tuning LLMs to enhance their long-context capabilities. We compare the Direct Preference Optimization (DPO) method with the Supervised Fine-Tuning (SFT) method, demonstrating DPO's superiority and data efficiency. Our experiments show that the fine-tuned model achieves a 4.04-point improvement over phi-3 and a 2.6\% increase on the Qasper benchmark using only 2000 samples. Despite facing limitations in data scale and processing costs, this study underscores the potential of DPO and high-quality data in advancing LLM performance. Additionally, the zero-shot benchmark results indicate that aggregated high-quality human reviews are overwhelmingly preferred over LLM-generated responses, even for the most capable models like GPT-4o. This suggests that high-quality human reviews are extremely rich in information, reasoning, and long-context retrieval, capabilities that even the most advanced models have not fully captured. These findings highlight the high utility of leveraging human reviews to further advance the field.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

↑