Understanding document images (e.g., invoices) is a core but challenging task since it requires complex functions such as reading text and a holistic understanding of the document. Current Visual Document Understanding (VDU) methods outsource the task of reading text to off-the-shelf Optical Character Recognition (OCR) engines and focus on the understanding task with the OCR outputs. Although such OCR-based approaches have shown promising performance, they suffer from 1) high computational costs for using OCR; 2) inflexibility of OCR models on languages or types of document; 3) OCR error propagation to the subsequent process. To address these issues, in this paper, we introduce a novel OCR-free VDU model named Donut, which stands for Document understanding transformer. As the first step in OCR-free VDU research, we propose a simple architecture (i.e., Transformer) with a pre-training objective (i.e., cross-entropy loss). Donut is conceptually simple yet effective. Through extensive experiments and analyses, we show a simple OCR-free VDU model, Donut, achieves state-of-the-art performances on various VDU tasks in terms of both speed and accuracy. In addition, we offer a synthetic data generator that helps the model pre-training to be flexible in various languages and domains. The code, trained model and synthetic data are available at https://github.com/clovaai/donut.
Handwritten Optical Character Recognition (OCR): A Comprehensive Systematic Literature Review (SLR)
Barrett Hodgson University（巴雷特霍奇森大学）；Quest International University（奎斯特国际大学）；NED University of Engineering and technology（NED工程技术大学）；Université Claude Bernard Lyon1（里昂第一大学）
Université Paris-Saclay, ENS Paris-Saclay, Centre Borelli, Gif-sur-Yvette, France（巴黎-萨克雷大学、巴黎-萨克雷高等师范学院、Borelli中心，法国）；IIE, Facultad de Ingenería, Universidad de la República（国际经济学院、工程学院，乌拉圭）；City University of Hong Kong, Kowloon, Hong Kong（香港城市大学，九龙，香港）；Determinant France, Paris, France（法国Determinant公司，巴黎，法国）；Sadjad University, Mashhad, Iran（沙贾尔大学，马什哈德，伊朗）
OCR for TIFF Compressed Document Images Directly in Compressed Domain Using Text segmentation and Hidden Markov Model
Indian Institute of Information Technology Allahabad（印度理工学院阿拉哈巴德分校）
在当今的技术时代，文档图像在我们的日常生活中发挥着重要和不可或缺的作用，特别是随着COVID-19的激增，数字扫描文档已成为沟通的主要来源，从而避免了任何形式的身体接触感染。扫描文档图像的存储和传输是一项非常占用内存的任务，因此压缩技术被用于在存档和传输之前减小图像大小。为了提取信息或对压缩图像进行操作，我们有两种方法：第一种方法是对图像进行压缩和操作，然后再次压缩图像，以提高存储和传输的效率。另一种方法是使用底层压缩算法的特性直接处理压缩形式的图像，而不涉及解压缩和重新压缩。本文提出了一种新的思路，即在压缩域中直接为CCITT（The International Telegraph and Telephone Consultative Committee，国际电报电话咨询委员会）压缩机打印的TIFF文档图像开发OCR。在将文本区域分割成行和词之后，使用CCITT的三种编码模式（水平、垂直和通过模式）应用HMM进行识别。实验结果表明，光学字符识别的通过模式给出了一个有前途的结果。
When Low CER is Not Enough: An Analysis of Hallucinations in Vision-Language OCR Systems on Historical Uruguayan Documents
Université Paris-Saclay（巴黎-萨克雷大学）；ENS Paris-Saclay（巴黎-萨克雷高等师范学校）；CNRS（法国国家科学研究中心）；Centre Borelli（博雷利中心）；Facultad de Ingeniería, Universidad de la República（乌拉圭共和国大学工程学院）；Lingnan University（岭南大学）
An End-to-End OCR Framework for Robust Arabic-Handwriting Recognition using a Novel Transformers-based Model and an Innovative 270 Million-Words Multi-Font Corpus of Classical Arabic with Diacritics
Multi-Type-TD-TSR -- Extracting Tables from Document Images using a Multi-stage Pipeline for Table Detection and Table Structure Recognition: from OCR to Structured Table Representations
