用于引文分析的合成与真实参考字符串，以及重新训练和样本外数据对有意义评估的重要性：使用GROBID，GIANT和Cora的实验
Synthetic vs. Real Reference Strings for Citation Parsing, and the Importance of Re-training and Out-Of-Sample Data for Meaningful Evaluations: Experiments with GROBID, GIANT and Cora
摘要：
引文解析，特别是深度神经网络，缺乏训练数据，因为可用的数据集通常只包含几千个训练实例。手动标记引用字符串非常耗时，因此综合创建的训练数据可能是一种解决方案。然而，到目前为止，还不知道合成创建的参考字符串是否适合训练用于引文解析的机器学习算法。为了找到答案，我们训练了Grobid，它使用了条件随机场，a）来自"真实"书目的人工标记的参考字符串，b）从GIANT数据集合成创建的参考字符串。我们发现，合成和有机参考字符串同样适用于训练Grobid（F1 = 0.74）。我们还发现，重新训练Grobid对其性能有显着影响，无论是合成数据还是真实数据（F1中为+30%）。在训练过程中尽可能多地使用标记字段也可以提高有效性，即使这些字段在评估数据中不可用（+13.5%F1）。我们的结论是，合成数据是适合训练（深）引文分析模型。我们进一步建议，在未来的评估参考解析器的评估数据相似和不相似的训练数据应用于更有意义的评估。
英文摘要：
Citation parsing, particularly with deep neural networks, suffers from a lack of training data as available datasets typically contain only a few thousand training instances. Manually labelling citation strings is very time-consuming, hence synthetically created training data could be a solution. However, as of now, it is unknown if synthetically created reference-strings are suitable to train machine learning algorithms for citation parsing. To find out, we train Grobid, which uses Conditional Random Fields, with a) human-labelled reference strings from 'real' bibliographies and b) synthetically created reference strings from the GIANT dataset. We find that both synthetic and organic reference strings are equally suited for training Grobid (F1 = 0.74). We additionally find that retraining Grobid has a notable impact on its performance, for both synthetic and real data (+30% in F1). Having as many types of labelled fields as possible during training also improves effectiveness, even if these fields are not available in the evaluation data (+13.5% F1). We conclude that synthetic data is suitable for training (deep) citation parsing models. We further suggest that in future evaluations of reference parsers both evaluation data similar and dissimilar to the training data should be used for more meaningful evaluations.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
