arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2403.02127cs.CVcs.AIcs.CL
收藏

LOCR:用于光学字符识别的位置引导 Transformer

LOCR: Location-Guided Transformer for Optical Character Recognition

Shanghai Artificial Intelligence Laboratory(上海人工智能实验室);Shanghai Jiaotong University(上海交通大学);University of Oxford(牛津大学)

Yu Sun, Dongzhan Zhou, Chen Lin, Conghui He, Wanli Ouyang, Han-Sen Zhong

AI总结:

本文提出 LOCR,在自回归 transformer 中引入位置引导,并利用大规模文本-位置对训练,显著提升学术与 OOD 文档 OCR 准确率、降低重复率,还支持基于位置提示的交互式识别。

摘要:

学术文档中充满了文本、公式、表格和图形,需要全面理解才能实现准确的光学字符识别(OCR)。尽管端到端 OCR 方法相比基于布局的方法具有更高准确率,但它们往往面临严重的重复问题,尤其是在域外(OOD)文档的复杂布局中。为解决该问题,我们提出 LOCR,这是一个在自回归过程中将位置引导集成到 transformer 架构中的模型。我们在包含来自 125K 个学术文档页面、超过 77M 个文本-位置对的数据集上训练该模型,数据包括单词、表格和数学符号的边界框。LOCR 能够熟练处理各种格式元素,并以 Markdown 语言生成内容。在我们基于 arXiv 构建的测试集上,以编辑距离、BLEU、METEOR 和 F-measure 衡量,LOCR 优于所有现有方法。LOCR 还将 arXiv 数据集中的重复页面频率从 4.4% 降至 0.5%,将 OOD 量子物理论文中的重复频率从 13.2% 降至 1.3%,并将 OOD 营销文档中的重复频率从 8.1% 降至 1.8%。此外,LOCR 具有交互式 OCR 模式,可通过人类提供的少量位置提示促进复杂文档的生成。

英文摘要:

Academic documents are packed with texts, equations, tables, and figures, requiring comprehensive understanding for accurate Optical Character Recognition (OCR). While end-to-end OCR methods offer improved accuracy over layout-based approaches, they often grapple with significant repetition issues, especially with complex layouts in Out-Of-Domain (OOD) documents.To tackle this issue, we propose LOCR, a model that integrates location guiding into the transformer architecture during autoregression. We train the model on a dataset comprising over 77M text-location pairs from 125K academic document pages, including bounding boxes for words, tables and mathematical symbols. LOCR adeptly handles various formatting elements and generates content in Markdown language. It outperforms all existing methods in our test set constructed from arXiv, as measured by edit distance, BLEU, METEOR and F-measure.LOCR also reduces repetition frequency from 4.4% of pages to 0.5% in the arXiv dataset, from 13.2% to 1.3% in OOD quantum physics documents and from 8.1% to 1.8% in OOD marketing documents. Additionally, LOCR features an interactive OCR mode, facilitating the generation of complex documents through a few location prompts from human.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑