arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2601.00567cs.IRcs.AI
收藏

利用学术概念索引改进科学文献检索

Improving Scientific Document Retrieval with Academic Concept Index

Korea University Seoul South Korea;University of Illinois Urbana-Champaign Champaign United States;Texas A\&M University College Station United States;Yonsei University Seoul South Korea;Pohang University of Science;Korea University;University of Illinois Urbana-Champaign;Texas A\&M University;Yonsei University

Jeyun Lee, Junhyoung Lee, Wonbin Kweon, Bowen Jin, Yu Zhang, Susik Yoon, Dongha Lee, Hwanjo Yu, Jiawei Han, Seongku Kang

AI总结:

针对通用检索器在科学领域因词汇和需求不匹配而表现不佳的问题,提出基于学术概念索引的方法,通过概念覆盖查询生成和概念聚焦上下文扩展,提升查询质量和检索性能。

摘要:

将通用领域的检索器适应到科学领域具有挑战性,原因在于缺乏大规模领域特定的相关性标注,以及词汇和信息需求的显著不匹配。最近的方法通过两个独立方向利用大型语言模型(LLMs)来解决这些问题:(1)生成合成查询以进行微调,(2)生成辅助上下文以支持相关性匹配。然而,这两个方向都忽略了科学文档中嵌入的多样化学术概念,常常产生冗余或概念狭窄的查询和上下文。为了解决这一限制,我们引入了一个学术概念索引,该索引从论文中提取关键概念,并在学术分类的指导下进行组织。这个结构化索引为改进这两个方向奠定了基础。首先,我们通过基于概念覆盖的查询生成(CCQGen)来增强合成查询生成,该方法自适应地以未覆盖的概念为条件,生成具有更广泛概念覆盖的互补查询。其次,我们通过概念聚焦的辅助上下文(CCExpand)来增强上下文增强,该方法利用一组文档片段作为对概念感知的CCQGen查询的简洁响应。大量实验表明,将学术概念索引纳入查询生成和上下文增强中,可以产生更高质量的查询、更好的概念对齐以及改进的检索性能。

英文摘要:

Adapting general-domain retrievers to scientific domains is challenging due to the scarcity of large-scale domain-specific relevance annotations and the substantial mismatch in vocabulary and information needs. Recent approaches address these issues through two independent directions that leverage large language models (LLMs): (1) generating synthetic queries for fine-tuning, and (2) generating auxiliary contexts to support relevance matching. However, both directions overlook the diverse academic concepts embedded within scientific documents, often producing redundant or conceptually narrow queries and contexts. To address this limitation, we introduce an academic concept index, which extracts key concepts from papers and organizes them guided by an academic taxonomy. This structured index serves as a foundation for improving both directions. First, we enhance the synthetic query generation with concept coverage-based generation (CCQGen), which adaptively conditions LLMs on uncovered concepts to generate complementary queries with broader concept coverage. Second, we strengthen the context augmentation with concept-focused auxiliary contexts (CCExpand), which leverages a set of document snippets that serve as concise responses to the concept-aware CCQGen queries. Extensive experiments show that incorporating the academic concept index into both query generation and context augmentation leads to higher-quality queries, better conceptual alignment, and improved retrieval performance.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑