将论文中的实验数据汇总使材料科学家能够构建更准确的性质预测模型并促进科学发现。最近,兴趣逐渐转向不仅提取单个材料属性,还提取整个实验测量。为了支持这一转变,我们介绍了LitXBench,一个用于评估从文献中提取实验方法的框架。我们还提出了LitXAlloy,一个包含1426个总测量数据来自19篇合金论文的密集基准。通过将基准的条目存储为Python对象,而不是基于文本的格式如CSV或JSON,我们提高了可审计性并启用了程序化数据验证。我们发现,前沿语言模型,如Gemini 3.1 Pro Preview,比现有的多轮提取管道高出多达0.37 F1。我们的结果表明,这种性能差距出现是因为提取管道将测量与组成而不是定义材料的加工步骤相关联。
英文摘要:
Aggregating experimental data from papers enables materials scientists to build better property prediction models and to facilitate scientific discovery. Recently, interest has grown in extracting not only single material properties but also entire experimental measurements. To support this shift, we introduce LitXBench, a framework for benchmarking methods that extract experiments from literature. We also present LitXAlloy, a dense benchmark comprising 1426 total measurements from 19 alloy papers. By storing the benchmark's entries as Python objects, rather than text-based formats such as CSV or JSON, we improve auditability and enable programmatic data validation. We find that frontier language models, such as Gemini 3.1 Pro Preview, outperform existing multi-turn extraction pipelines by up to 0.37 F1. Our results suggest that this performance gap arises because extraction pipelines associate measurements with compositions rather than the processing steps that define a material.
EmpiriGraph-Psy: A Dataset and LLM Pipeline for Extracting Empirical Relation Graphs from Psychology Abstracts
University of Warwick(华威大学);Mathematical Sciences Institute, The Australian National University(澳大利亚国立大学数学科学研究所);School of Computer Science, University of Sheffield(谢菲尔德大学计算机学院)
Extracting a Knowledge Base of Mechanisms from COVID-19 Papers
University of Washington(华盛顿大学);Allen Institute for Artificial Intelligence(艾伦人工智能研究所);Swedish Medical Group(瑞典医疗集团);Microsoft Research(微软研究院);The Hebrew University of Jerusalem(耶路撒冷希伯来大学)
Beyond Text and Tables: Vision-Language Model Integration in ComProScanner for Extracting Materials Data from Scientific Figures with High Accuracy
Energy, Materials and Environment Research Centre, London South Bank University, London SE1 0AA, UK(能源、材料与环境研究中心,伦敦南银行大学);School of Engineering and Design, London South Bank University, London SE1 0AA, UK(工程与设计学院,伦敦南银行大学);Bioscience and Bioengineering Research Centre, London South Bank University, London SE1 0AA, UK(生物科学与生物工程研究中心,伦敦南银行大学);Department of Physics, Kings College London, London WC2R 2LS, UK(物理系,伦敦国王学院)
Comparison of Feature Learning Methods for Metadata Extraction from PDF Scholarly Documents
Fraunhofer Institute for Applied Information Technology FIT(弗劳恩霍夫应用信息技术研究所FIT);University Hospital of Cologne(科隆大学医院);University of Koblenz(科布伦茨大学);Soochow University(苏州大学)
Extracting Variable-Depth Logical Document Hierarchy from Long Documents: Method, Evaluation, and Application
Institute of Computing Technology, Chinese Academy of Sciences(中国科学院计算技术研究所);University of Chinese Academy of Sciences(中国科学院大学);Tencent Holdings Ltd.(腾讯控股有限公司);Peng Cheng Laboratory(鹏城实验室)
在本文中,我们解决了从专利中提取科学参考文献的挑战。我们将这个问题作为一个序列标记任务来处理,并研究了BERT模型在提取这些长序列方面的优点。专利中对科学文献的引用与研究科学与工业之间的联系有关。大多数先前的工作只使用这种分析的头版引文,这些引文在专利档案的元数据中提供。在本文中,我们建立在以前的工作,使用条件随机场(CRF)和天赋的参考提取。我们提高了训练数据的质量,并在标记数据上训练了三个基于BERT的模型(BERT,bioBERT,sciBERT)。我们发现,改进的训练数据导致训练模型的质量有很大的提高。此外,BERT模型击败了CRF和Flair,通过交叉验证获得了约97%的召回分数。使用最佳模型,我们标记了大量的33000项专利,提取引用,并将它们与Web of Science数据库中的出版物进行匹配。我们提取了比旧的训练数据和方法多50%的参考文献:总共73.5万个参考文献。有了这些专利-出版链接,后续研究将进一步分析哪些类型的科学工作导致发明。