arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2406.05348cs.CLcond-mat.mtrl-scics.AIcs.IR
收藏

迈向可靠的临时科学信息抽取:基于两个材料数据集的案例研究

Toward Reliable Ad-hoc Scientific Information Extraction: A Case Study on Two Materials Datasets

University of New Hampshire(新罕布什尔大学);University of California, Santa Barbara(加州大学圣芭芭拉分校)

Satanu Ghosh, Neal R. Brodnik, Carolina Frey, Collin Holgate, Tresa M. Pollock, Samantha Daly, Samuel Carton

AI总结:

该研究探究GPT-4从科学文献中执行基于临时模式的信息抽取能力,通过基础提示方法复现两个材料科学数据集,经人工错误分析明确模型难点并提出研究方向以提升该任务性能。

摘要:

我们探究了GPT-4从科学文献中执行基于临时模式的信息抽取的能力。具体评估其能否通过基础提示方法,利用原始手动抽取数据集时所用的手稿,复现两个现有材料科学数据集。我们聘请材料科学家进行详细的人工错误分析,以评估模型在忠实抽取所需信息时的难点,并基于他们的见解提出研究方向,以解决这一具有广泛重要性的任务。

英文摘要:

We explore the ability of GPT-4 to perform ad-hoc schema based information extraction from scientific literature. We assess specifically whether it can, with a basic prompting approach, replicate two existing material science datasets, given the manuscripts from which they were originally manually extracted. We employ materials scientists to perform a detailed manual error analysis to assess where the model struggles to faithfully extract the desired information, and draw on their insights to suggest research directions to address this broadly important task.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑