arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2603.25640cs.DLcs.CL
收藏

RenoBench: 引文解析基准

RenoBench: A Citation Parsing Benchmark

Graduate School of Education, Stanford University(斯坦福大学教育研究生院);Public Knowledge Project, Simon Fraser University(公共知识项目,西蒙弗雷泽大学);DataCite, Hannover, Germany(DataCite,德国汉诺威);California Digital Library, University of California Office of the President(加州数字图书馆,加州大学校长办公室)

Parth Sarin, Juan Pablo Alperin, Adam Buttrick, Dione Mentis

AI总结:

针对现有引文解析评估方法不可泛化、基于合成数据或不可公开获取的问题,提出从四个出版生态系统收集的公开基准RenoBench,通过自动验证和特征采样构建多语言、多类型数据集,并评估多种解析系统,结果表明语言模型(尤其微调后)表现优异,为可重复标准化评估奠定基础。

摘要:

准确解析引文对于机器可读的学术基础设施是必要的。但是,尽管对该问题持续关注,现有的评估技术通常不可泛化、基于合成数据或不可公开获取。我们引入了RenoBench,一个用于引文解析的公共领域基准,来源于四个出版生态系统(SciELO、Redalyc、Public Knowledge Project和Open Research Europe)发布的PDF。从161,000条带注释的引文开始,我们应用自动验证和基于特征的采样,生成了一个包含10,000条引文的数据集,涵盖多种语言、出版类型和平台。然后,我们评估了多种引文解析系统,并报告了字段级别的精确率和召回率。我们的结果显示语言模型表现强劲,尤其是在微调后。RenoBench实现了对引文解析系统的可重复、标准化评估,并为推进自动化引文解析和元科学研究提供了基础。

英文摘要:

Accurate parsing of citations is necessary for machine-readable scholarly infrastructure. But, despite sustained interest in this problem, existing evaluation techniques are often not generalizable, based on synthetic data, or not publicly available. We introduce RenoBench, a public domain benchmark for citation parsing, sourced from PDFs released on four publishing ecosystems: SciELO, Redalyc, the Public Knowledge Project, and Open Research Europe. Starting from 161,000 annotated citations, we apply automated validation and feature-based sampling to produce a dataset of 10,000 citations spanning multiple languages, publication types, and platforms. We then evaluate a variety of citation parsing systems and report field-level precision and recall. Our results show strong performance from language models, particularly when fine-tuned. RenoBench enables reproducible, standardized evaluation of citation parsing systems, and provides a foundation for advancing automated citation parsing and metascientific research.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑