arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2407.02409cs.CL
收藏

基于 LLM 的排行榜生成中的有效上下文选择:一项实证研究

Effective Context Selection in LLM-based Leaderboard Generation: An Empirical Study

L3S Research Center(L3S研究中心);Leibniz University of Hannover(汉诺威莱布尼茨大学);TIB Leibniz Information Centre for Science and Technology(莱布尼茨科技信息中心(TIB))

Salomon Kabongo, Jennifer D'Souza, Sören Auer

AI总结:

本文研究 LLM 生成 AI 研究排行榜时的上下文选择问题,提出基于 FLAN-T5 指令微调的文本生成方法,并通过三类上下文实验证明其能提升准确率、减少幻觉。

摘要:

本文探讨了上下文选择对大型语言模型(LLM)在生成人工智能(AI)研究排行榜时效率的影响;该任务被定义为从学术论文中抽取(Task, Dataset, Metric, Score)四元组。通过将这一挑战构建为文本生成目标,并使用 FLAN-T5 系列进行指令微调,我们提出了一种新方法。该方法无需预定义分类体系,即可适应新的发展,并在这方面超越了传统的自然语言推理(NLI)方法。通过对三种具有不同选择性和长度的不同上下文类型进行实验,本研究证明了有效上下文选择在提高 LLM 准确率和减少幻觉方面的重要性,为可靠且高效地生成 AI 排行榜提供了一条新途径。这一贡献不仅推进了排行榜生成领域的当前最优水平,也为缓解基于 LLM 的信息抽取中的常见挑战阐明了策略。

英文摘要:

This paper explores the impact of context selection on the efficiency of Large Language Models (LLMs) in generating Artificial Intelligence (AI) research leaderboards, a task defined as the extraction of (Task, Dataset, Metric, Score) quadruples from scholarly articles. By framing this challenge as a text generation objective and employing instruction finetuning with the FLAN-T5 collection, we introduce a novel method that surpasses traditional Natural Language Inference (NLI) approaches in adapting to new developments without a predefined taxonomy. Through experimentation with three distinct context types of varying selectivity and length, our study demonstrates the importance of effective context selection in enhancing LLM accuracy and reducing hallucinations, providing a new pathway for the reliable and efficient generation of AI leaderboards. This contribution not only advances the state of the art in leaderboard generation but also sheds light on strategies to mitigate common challenges in LLM-based information extraction.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑