arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2508.01643cs.IRcs.CL
收藏

ChEmbed:通过领域特定文本嵌入增强化学文献检索

ChEmbed: Enhancing Chemical Literature Search Through Domain-Specific Text Embeddings

Department of Computational Science and Engineering(计算科学与工程系);McMaster University(麦斯特大学);BASF Canada Inc(巴斯夫加拿大公司);BASF Corporation(巴斯夫公司)

Ali Shiraee Kasmaee, Mohammad Khodadad, Mahdi Astaraki, Mohammad Arshi Saloot, Nicholas Sherck, Hamidreza Mahyar, Soheila Samiee

AI总结:

ChEmbed通过领域自适应嵌入模型,利用170万合成查询-段落对和900个化学专用标记,在ChemRxiv基准上将nDCG@10从0.82提升至0.91,有效改善化学文献检索。

摘要:

检索增强生成(RAG)系统在化学领域高度依赖于对化学文献的准确且相关的检索。然而,通用文本嵌入模型常常无法充分表示复杂的化学术语,导致检索质量欠佳。专门针对化学文献检索定制的嵌入模型尚未开发,留下了显著的性能差距。为解决这一挑战,我们引入了ChEmbed,一个基于PubChem、Semantic Scholar和ChemRxiv语料库中化学特定文本数据集进行微调的领域自适应文本嵌入模型系列。为了创建有效的训练数据,我们利用大语言模型合成生成查询,产生了约170万个高质量的查询-段落对。此外,我们通过向先前未使用的槽位添加900个化学专用标记来增强分词器,这显著减少了化学实体(如IUPAC名称)的碎片化。ChEmbed还保持了8192个标记的上下文长度,与许多其他通常具有512或2048个标记上下文长度的开源嵌入模型相比,能够高效检索更长的段落。在我们新引入的ChemRxiv检索基准上进行评估,ChEmbed优于最先进的通用嵌入模型,将nDCG@10从0.82提高到0.91(+9个百分点)。ChEmbed代表了一种实用、轻量级且可复现的嵌入解决方案,有效改善了化学文献检索的性能。

英文摘要:

Retrieval-Augmented Generation (RAG) systems in chemistry heavily depend on accurate and relevant retrieval of chemical literature. However, general-purpose text embedding models frequently fail to adequately represent complex chemical terminologies, resulting in suboptimal retrieval quality. Existing embedding models for chemistry are outdated, and none is tailored to chemical literature retrieval, leaving a substantial performance gap. To address this challenge, we introduce ChEmbed, the first purpose-built family of domain-adapted text embedding models engineered for chemical literature retrieval. These models are fine-tuned via contrastive learning on a dataset comprising chemistry-specific text from the PubChem, Semantic Scholar, and ChemRxiv corpora. To create effective training data, we employ large language models to synthetically generate queries, resulting in approximately 1.7 million high-quality query-passage pairs. Additionally, we augment the tokenizer by adding 900 chemically specialized tokens to previously unused slots, which reduces the fragmentation of chemical entities, such as IUPAC names. ChEmbed also maintains an 8192-token context length, enabling retrieval of longer passages than many open-source embedding models allow. Evaluated on our newly introduced ChemRxiv Retrieval benchmark, ChEmbed outperforms state-of-the-art general embedding models, raising MRR@10 from 0.781 to 0.882 (+10.1 pp). It also substantially outperforms domain-specific embedding models such as Chemical-BERT, improving MRR@10 from 0.096 to 0.882. A role-based retrieval analysis using PubChem descriptions and ChEBI annotations shows that the improvement extends to chemical-role queries. ChEmbed represents a practical, lightweight, and reproducible embedding solution that effectively improves chemical literature retrieval.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑