arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2601.14949cs.IR
收藏

我应该引用什么?一个用于学术引用预测的RAG基准

What Should I Cite? A RAG Benchmark for Academic Citation Prediction

Tsinghua University(清华大学);University of Science and Technology of China(中国科学技术大学);Beijing Institute of Technology(北京理工大学);Peking University(北京大学);Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所)
Leqi Zheng, Jiajun Zhang, Canzhi Chen, Chaokun Wang, Hongwei Li, Yuying Li, Yaoxin Mao, Shannan Yan, Zixin Song, Zhiyuan Feng, Zhaolu Kang, Zirong Chen, Hang Zh… 展开作者

Leqi Zheng, Jiajun Zhang, Canzhi Chen, Chaokun Wang, Hongwei Li, Yuying Li, Yaoxin Mao, Shannan Yan, Zixin Song, Zhiyuan Feng, Zhaolu Kang, Zirong Chen, Hang Zhang, Qiang Liu, Liang Wang, Ziyang Liu

AI总结:

CiteRAG提出一个用于学术引用预测的RAG基准,通过多层次检索策略和生成器,评估大型语言模型在引用预测中的性能。

摘要:

随着基于网络的学术出版物的快速增长,每年发表的论文越来越多,使找到相关先前工作变得更加困难。引用预测旨在自动建议合适的参考文献,帮助学者在不断扩大的科学文献中导航。在这里,我们介绍了CiteRAG,这是第一个综合检索增强生成(RAG)集成的基准,用于评估大型语言模型在学术引用预测上的能力,具有多层次检索策略、专门的检索器和生成器。我们的基准做出了四个核心贡献:(1)我们建立了两个不同的粒度的引用预测任务实例。任务1专注于粗粒度的列表特定引用预测,而任务2则针对细粒度的位置特定引用预测。为了增强这两个任务,我们构建了一个包含7,267个实例的任务1数据集和8,541个实例的任务2数据集,从而能够全面评估检索和生成。 (2)我们构建了一个三级大规模语料库,包含554,000篇论文,覆盖许多主要子领域,使用增量流程。 (3)我们提出了一种多层次的混合RAG方法用于引用预测,通过对比学习微调嵌入模型以捕捉复杂的引用关系,配以专门的生成模型。 (4)我们在最先进的语言模型上进行了广泛的实验,包括闭源API、开源模型和我们微调的生成器,展示了我们框架的有效性。我们的开源工具包使评估可重复,并专注于学术文献,提供了引用预测的第一个全面评估框架,并为其他科学领域提供了方法论模板。我们的源代码和数据已发布在https://github.com/LQgdwind/CiteRAG。

英文摘要:

With the rapid growth of Web-based academic publications, more and more papers are being published annually, making it increasingly difficult to find relevant prior work. Citation prediction aims to automatically suggest appropriate references, helping scholars navigate the expanding scientific literature. Here we present \textbf{CiteRAG}, the first comprehensive retrieval-augmented generation (RAG)-integrated benchmark for evaluating large language models on academic citation prediction, featuring a multi-level retrieval strategy, specialized retrievers, and generators. Our benchmark makes four core contributions: (1) We establish two instances of the citation prediction task with different granularity. Task 1 focuses on coarse-grained list-specific citation prediction, while Task 2 targets fine-grained position-specific citation prediction. To enhance these two tasks, we build a dataset containing 7,267 instances for Task 1 and 8,541 instances for Task 2, enabling comprehensive evaluation of both retrieval and generation. (2) We construct a three-level large-scale corpus with 554k papers spanning many major subfields, using an incremental pipeline. (3) We propose a multi-level hybrid RAG approach for citation prediction, fine-tuning embedding models with contrastive learning to capture complex citation relationships, paired with specialized generation models. (4) We conduct extensive experiments across state-of-the-art language models, including closed-source APIs, open-source models, and our fine-tuned generators, demonstrating the effectiveness of our framework. Our open-source toolkit enables reproducible evaluation and focuses on academic literature, providing the first comprehensive evaluation framework for citation prediction and serving as a methodological template for other scientific domains. Our source code and data are released at https://github.com/LQgdwind/CiteRAG.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑