arXivDaily arXiv每日学术速递 周一至周五更新
​通知:联邦节假日调休:哥伦布日 / 原住民日(Columbus Day / Indigenous Peoples' Day),今日(13日)无更新。
arXiv 2011.03502cs.CLcs.LG

An Unsupervised method for OCR Post-Correction and Spelling Normalisation for Finnish

University of Helsinki(赫尔辛基大学);Rootroo Ltd(Rootroo有限公司);University of Gothenburg(哥德堡大学)

Quan Duong, Mika Hämäläinen, Simon Hengchen

英文摘要:

Historical corpora are known to contain errors introduced by OCR (optical character recognition) methods used in the digitization process, often said to be degrading the performance of NLP systems. Correcting these errors manually is a time-consuming process and a great part of the automatic approaches have been relying on rules or supervised machine learning. We build on previous work on fully automatic unsupervised extraction of parallel data to train a character-based sequence-to-sequence NMT (neural machine translation) model to conduct OCR error correction designed for English, and adapt it to Finnish by proposing solutions that take the rich morphology of the language into account. Our new method shows increased performance while remaining fully unsupervised, with the added benefit of spelling normalisation. The source code and models are available on GitHub and Zenodo.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑