arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2603.23849cs.IR
收藏

VILLA:利用大语言模型从科学文献中进行多用途信息检索

VILLA: Versatile Information Retrieval From Scientific Literature Using Large LAnguage Models

Virginia Tech(弗吉尼亚理工大学);University of Chicago(芝加哥大学);University of Michigan(密歇根大学);Cornell University(康奈尔大学)

Blessy Antony, Amartya Dutta, Sneha Aggarwal, Vasu Gatne, Ozan Gökdemir, Samantha Grimes, Adam Lauring, Brian R. Wasik, Anuj Karpatne, T. M. Murali

AI总结:

本文提出VILLA框架,通过构建病毒突变提取任务,解决复杂开放任务中科学信息提取的挑战,展示其在信息检索中的优越性能。

摘要:

缺乏高质量的真实数据集阻碍了机器学习模型的训练,限制了人工智能在科学研究中的潜力。利用大语言模型(LLM)从文献中提取科学信息已成为自动化创建这些数据集的有效方法。然而,现有的基于LLM的方法和评估研究主要集中在生物医学和化学等广泛主题上,局限于选择性任务,并且专注于从短且格式良好的文本中提取信息。在复杂、开放性任务中,科学信息提取方法的潜力尚未得到充分探索。本文通过一个几乎被忽视的领域——病毒学,来填补这些研究空白。我们设计了一个独特的、开放性科学信息提取任务,即提取特定病毒中改变其与宿主相互作用的突变。我们开发了一个新的、多步骤检索增强生成(RAG)框架,称为VILLA,用于科学信息提取。同时,我们整理了一个包含来自239篇科学论文的629个突变数据集,用于突变提取任务的地面真实数据。最后,我们通过一种新颖且全面的评估和与其他最先进的RAG和基于代理的工具进行比较,展示了VILLA在信息检索中的优越性能。

英文摘要:

The lack of high-quality ground truth datasets to train machine learning (ML) models impedes the potential of artificial intelligence (AI) for science research. Scientific information extraction (SIE) from the literature using LLMs is emerging as a powerful approach to automate the creation of these datasets. However, existing LLM-based approaches and benchmarking studies for SIE focus on broad topics such as biomedicine and chemistry, are limited to choice-based tasks, and focus on extracting information from short and well-formatted text. The potential of SIE methods in complex, open-ended tasks is considerably under-explored. In this study, we used a domain that has been virtually ignored in SIE, namely virology, to address these research gaps. We design a unique, open-ended SIE task of extracting mutations in a given virus that modify its interaction with the host. We develop a new, multi-step retrieval augmented generation (RAG) framework called VILLA for SIE. In parallel, we curate a novel dataset of 629 mutations in ten influenza A virus proteins obtained from 239 scientific publications to serve as ground truth for the mutation extraction task. Finally, we demonstrate VILLA's superior performance using a novel and comprehensive evaluation and comparison with vanilla RAG and other state-of-the art RAG- and agent-based tools for SIE.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑