arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2011.05402cs.CL
收藏

OCR濒危语言文本的后期纠正

OCR Post Correction for Endangered Language Texts

Carnegie Mellon University(卡内基梅隆大学);George Mason University(乔治梅森大学)

Shruti Rijhwani, Antonios Anastasopoulos, Graham Neubig

摘要:

对于大多数濒危语言,几乎没有数据可用于构建自然语言处理模型。但是,这些语言的文本数据通常以不可机器读取的格式存在,例如纸质书籍和扫描图像。在这项工作中,我们解决了从这些资源中提取文本的任务。我们创建了一个基准数据集的transcription扫描书籍在三个极度濒危的语言,并提出了一个系统的分析,如何通用OCR工具是不强大的濒危语言的数据稀缺的设置。我们开发了一种OCR后校正方法,以减轻在这种数据稀缺的环境中的训练,降低了三种语言的识别出错率平均34%。

英文摘要:

There is little to no data available to build natural language processing models for most endangered languages. However, textual data in these languages often exists in formats that are not machine-readable, such as paper books and scanned images. In this work, we address the task of extracting text from these resources. We create a benchmark dataset of transcriptions for scanned books in three critically endangered languages and present a systematic analysis of how general-purpose OCR tools are not robust to the data-scarce setting of endangered languages. We develop an OCR post-correction method tailored to ease training in this data-scarce setting, reducing the recognition error rate by 34% on average across the three languages.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑