古老但已数字化:通过创建 KHAMIS 数据集开发东叙利亚文手写光学字符识别
Ancient but Digitized: Developing Handwritten Optical Character Recognition for East Syriac Script Through Creating KHAMIS Dataset
AI总结:
本文针对濒危的东叙利亚文,创建了包含624句手写句子的KHAMIS数据集,并以此微调Tesseract-OCR引擎的预训练模型,成功开发了手写OCR模型,其测试集表现比默认模型好两倍。
摘要:
许多语言拥有大量手写文本,例如关于民间故事和历史叙事的古代手稿,或当代文档与信件。这些文本的数字化具有广泛的应用,如日常任务、文化研究和历史研究。叙利亚语是一种古老、濒危且低资源的语言,尚未得到其所需和应得的关注。本文报告了一个研究项目,旨在基于手写叙利亚语文本开发光学字符识别(OCR)模型,作为为这种濒危语言构建更多数字服务的起点。我们创建了一个数据集 KHAMIS(受东叙利亚语诗人 Khamis bar Qardahe 启发),由东叙利亚文手写句子组成。我们用它对 Tesseract-OCR 引擎的预训练叙利亚语模型在手写数据上进行了微调。数据从能够读写该语言的志愿者处收集以创建 KHAMIS。KHAMIS 目前包含从 31 名大学生和 1 位教授处收集的 624 句手写叙利亚语句子,它将部分在线提供,并在不久的将来提供完整数据集用于开发和研究。结果,该手写 OCR 模型在训练集和评估集上的字符错误率分别为 1.097-1.610% 和 8.963-10.490%,在测试集上评估时字符错误率为 18.89-19.71%,词错误率为 62.83-65.42%,这比 Tesseract 的默认叙利亚语模型好两倍。
英文摘要:
Many languages have vast amounts of handwritten texts, such as ancient scripts about folktale stories and historical narratives or contemporary documents and letters. Digitization of those texts has various applications, such as daily tasks, cultural studies, and historical research. Syriac is an ancient, endangered, and low-resourced language that has not received the attention it requires and deserves. This paper reports on a research project aimed at developing a optical character recognition (OCR) model based on the handwritten Syriac texts as a starting point to build more digital services for this endangered language. A dataset was created, KHAMIS (inspired by the East Syriac poet, Khamis bar Qardahe), which consists of handwritten sentences in the East Syriac script. We used it to fine-tune the Tesseract-OCR engine's pretrained Syriac model on handwritten data. The data was collected from volunteers capable of reading and writing in the language to create KHAMIS. KHAMIS currently consists of 624 handwritten Syriac sentences collected from 31 university students and one professor, and it will be partially available online and the whole dataset available in the near future for development and research purposes. As a result, the handwritten OCR model was able to achieve a character error rate of 1.097-1.610% and 8.963-10.490% on both training and evaluation sets, respectively, and both a character error rate of 18.89-19.71% and a word error rate of 62.83-65.42% when evaluated on the test set, which is twice as better than the default Syriac model of Tesseract.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。