Recent advances in OCR have shown that an end-to-end (E2E) training pipeline that includes both detection and recognition leads to the best results. However, many existing methods focus primarily on Latin-alphabet languages, often even only case-insensitive English characters. In this paper, we propose an E2E approach, Multiplexed Multilingual Mask TextSpotter, that performs script identification at the word level and handles different scripts with different recognition heads, all while maintaining a unified loss that simultaneously optimizes script identification and multiple recognition heads. Experiments show that our method outperforms the single-head model with similar number of parameters in end-to-end recognition tasks, and achieves state-of-the-art results on MLT17 and MLT19 joint text detection and script identification benchmarks. We believe that our work is a step towards the end-to-end trainable and scalable multilingual multi-purpose OCR system. Our code and model will be released.
We describe a novel line-level script identification method. Previous work repurposed an OCR model generating per-character script codes, counted to obtain line-level script identification. This has two shortcomings. First, as a sequence-to-sequence model it is more complex than necessary for the sequence-to-label problem of line script identification. This makes it harder to train and inefficient to run. Second, the counting heuristic may be suboptimal compared to a learned model. Therefore we reframe line script identification as a sequence-to-label problem and solve it using two components,…
ReceiptSense: Beyond Traditional OCR -- A Dataset for Receipt Understanding
Innsbruck University（因斯布鲁克大学）；High Institute for Computers（计算机与管理信息高级学院）；Chungbuk National University（Chungbuk国立大学）；Louisville University（路易斯维尔大学）；University of Innsbruck（因斯布鲁克大学）
All-in-One Multilingual Scene Text Recognition with Script-aware Mixture-of-Experts
Institute of Trustworthy Embodied AI, Fudan University（复旦大学可信具身智能研究院）；Shanghai Key Laboratory of Multimodal Embodied AI（上海多模态具身智能重点实验室）；WeChat Vision, Tencent Inc.（腾讯微信视觉团队）；South China University of Technology（华南理工大学）
India is a multilingual multi-script country. In every state of India there are two languages one is state local language and the other is English. For example in Andhra Pradesh, a state in India, the document may contain text words in English and Telugu script. For Optical Character Recognition (OCR) of such a bilingual document, it is necessary to identify the script before feeding the text words to the OCRs of individual scripts. In this paper, we are introducing a simple and efficient technique of script identification for Kannada, English and Hindi text words of a printed document. The pr…
HunyuanOCR-1.5: Making Lightweight OCR VLMs Faster and Better
Institute of Information Engineering, Chinese Academy of Sciences（中国科学院信息工程研究所）；Large Language Model Department, Tencent（腾讯大语言模型部）；Nankai University（南开大学）
研究改进轻量级端到端OCR视觉语言模型。基于HunyuanOCR-1.0架构，用DFlash提升效率，提出Agentic Data Flow增强能力，在多任务表现出色，还将发布模型权重和代码。
Most existing OCR methods focus on alphanumeric characters due to the popularity of English and numbers, as well as their corresponding datasets. On extending the characters to more languages, recent methods have shown that training different scripts with different recognition heads can greatly improve the end-to-end recognition accuracy compared to combining characters from all languages in the same recognition head. However, we postulate that similarities between some languages could allow sharing of model parameters and benefit from joint training. Determining language groupings, however, i…
Evaluating OCR performance on food packaging labels in South Africa
Department of Computer Science, University of the Western Cape（计算机科学系，南非西部大学）；School of Public Health, University of the Western Cape（公共卫生学院，南非西部大学）；Department of Dietetics and Nutrition, University of the Western Cape（营养学系，南非西部大学）；eResearch Office, University of the Western Cape（eResearch办公室，南非西部大学）
OCRBench: On the Hidden Mystery of OCR in Large Multimodal Models
Huazhong University of Science and Technology（华中科技大学）；Microsoft Research（微软研究院）；University of Science and Technology Beijing（北京科技大学）；Chinese Academy of Sciences（中国科学院）；South China University of Technology（华南理工大学）
Many studies on (Offline) Handwritten Text Recognition (HTR) systems have focused on building state-of-the-art models for line recognition on small corpora. However, adding HTR capability to a large scale multilingual OCR system poses new challenges. This paper addresses three problems in building such systems: data, efficiency, and integration. Firstly, one of the biggest challenges is obtaining sufficient amounts of high quality training data. We address the problem by using online handwriting data collected for a large scale production online handwriting recognition system. We describe our …
Systematic Literature Review of Machine Learning Models and Applications for Text Recognition
Faculty of Electrical Engineering, Universiti Teknologi Malaysia（马来西亚工艺大学电气工程学院）；Faculty of Artificial Intelligence, Universiti Teknologi Malaysia（马来西亚工艺大学人工智能学院）；Department of Electrical and Electronic Engineering, American International University-Bangladesh（孟加拉国美国国际大学电气与电子工程系）；Department of Electrical Engineering, Balochistan University of Information Technology, Engineering and Management Sciences（巴基斯坦俾路支省信息技术、工程与管理科学大学电气工程系）
In a multilingual country like India where 12 different official scripts are in use, automatic identification of handwritten script facilitates many important applications such as automatic transcription of multilingual documents, searching for documents on the web/digital archives containing a particular script and for the selection of script specific Optical Character Recognition (OCR) system in a multilingual environment. In this paper, we propose a robust method towards identifying scripts from the handwritten documents at text line-level. The recognition is based upon features extracted u…
Benchmarking and Boosting Multilingual Capabilities of LVLMs via OCR-Centric Reinforcement Learning
Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）；Shanghai Jiao Tong University（上海交通大学）；Peking University（北京大学）；Sun Yat-Sen University（中山大学）；Chinese University of Hong Kong（香港中文大学）
DDVR-Net: Degradation-Aware Decoupling and Visibility-Guided Restoration for Historical Document Recognition
Institute of National Security, Minzu University of China（中央民族大学国家安全研究院）；School of Information Engineering, Minzu University of China（中央民族大学信息工程学院）；School of Artificial Intelligence and Computer Science, North China University of Technology（北方工业大学人工智能与计算机学院）
Automatic identification of script is an essential component of a multilingual OCR engine. In this paper, we present an efficient, lightweight, real-time and on-device spatial attention based CNN-LSTM network for scene text script identification, feasible for deployment on resource constrained mobile devices. Our network consists of a CNN, equipped with a spatial attention module which helps reduce the spatial distortions present in natural images. This allows the feature extractor to generate rich image representations while ignoring the deformities and thereby, enhancing the performance of t…
