历史印刷品的OCR及其在建立历时语料库中的应用:使用RIDGES草药语料库的案例研究
OCR of historical printings with an application to building diachronic corpora: A case study using the RIDGES herbal corpus
摘要:
本文描述了一个案例研究的结果,该案例研究将基于神经网络的光学字符识别(OCR)应用于1487年至1870年印刷的书籍的扫描图像,方法是在RIDGES草药文本语料库[@OdebrechtEtAlSubmitted]上训练OCR引擎OCRopus [@breuel2013high]。训练特定的OCR模型是可能的,因为必要的 * 地面真相 * 可以作为纠错的外交翻译。OCR结果已被评估的准确性对地面真相看不见的测试集。单个文档的机器可读文本的字符和单词准确度(正确识别项目的百分比)范围从94%到99%以上(字符级别)和从76%到97%(单词级别)。这包括最早的印刷书籍,直到最近才被认为是OCR方法无法访问的。此外,在由不同印刷日期和不同排版的书籍组成的语料库的一部分上训练的OCR模型 *(混合模型)* 已经测试了它们对包含其他字体的另一部分的书籍的预测能力,大多数产生的字符准确率远高于90%。因此,似乎有可能构建在一系列字体上训练的通用模型,这些模型可以应用于各种各样的历史印刷品,仍然可以给出良好的结果。然后,对某些页面进行适度的后校正工作将使个别模型的训练具有更好的准确性。使用这种方法,包括早期印刷品的历时语料库可以比手工转录更快,更便宜地构建。这里报道的OCR方法打开了通过基本上自动化的手段将我们的印刷文本文化遗产转化为电子文本的可能性,这是大规模转换扫描书籍的先决条件。
英文摘要:
This article describes the results of a case study that applies Neural Network-based Optical Character Recognition (OCR) to scanned images of books printed between 1487 and 1870 by training the OCR engine OCRopus [@breuel2013high] on the RIDGES herbal text corpus [@OdebrechtEtAlSubmitted]. Training specific OCR models was possible because the necessary *ground truth* is available as error-corrected diplomatic transcriptions. The OCR results have been evaluated for accuracy against the ground truth of unseen test sets. Character and word accuracies (percentage of correctly recognized items) for the resulting machine-readable texts of individual documents range from 94% to more than 99% (character level) and from 76% to 97% (word level). This includes the earliest printed books, which were thought to be inaccessible by OCR methods until recently. Furthermore, OCR models trained on one part of the corpus consisting of books with different printing dates and different typesets *(mixed models)* have been tested for their predictive power on the books from the other part containing yet other fonts, mostly yielding character accuracies well above 90%. It therefore seems possible to construct generalized models trained on a range of fonts that can be applied to a wide variety of historical printings still giving good results. A moderate postcorrection effort of some pages will then enable the training of individual models with even better accuracies. Using this method, diachronic corpora including early printings can be constructed much faster and cheaper than by manual transcription. The OCR methods reported here open up the possibility of transforming our printed textual cultural heritage into electronic text by largely automatic means, which is a prerequisite for the mass conversion of scanned books.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。