Handwriting Recognition in Low-resource Scripts using Adversarial Learning
Nanyang Technological University（南洋理工大学）；Institute of Engineering & Management（工程与管理学院）；Jadavpur University（贾达普大学）；Indian Institute of Technology Roorkee（印度理工学院鲁尔基分校）
Ayan Kumar Bhunia, Abhirup Das, Ankan Kumar Bhunia, Perla Sai Raj Kishore, Partha Pratim Roy
备注：
IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), 2019
Handwritten Word Recognition and Spotting is a challenging field dealing with handwritten text possessing irregular and complex shapes. The design of deep neural network models makes it necessary to extend training datasets in order to introduce variations and increase the number of samples; word-retrieval is therefore very difficult in low-resource scripts. Much of the existing literature comprises preprocessing strategies which are seldom sufficient to cover all possible variations. We propose the Adversarial Feature Deformation Module (AFDM) that learns ways to elastically warp extracted features in a scalable manner. The AFDM is inserted between intermediate layers and trained alternatively with the original framework, boosting its capability to better learn highly informative features rather than trivial ones. We test our meta-framework, which is built on top of popular word-spotting and word-recognition frameworks and enhanced by the AFDM, not only on extensive Latin word datasets but also sparser Indic scripts. We record results for varying training data sizes, and observe that our enhanced network generalizes much better in the low-data regime; the overall word-error rates and mAP scores are observed to improve as well.
Language-Agnostic Visual Embeddings for Cross-Script Handwriting Retrieval
School of Integrated Circuits, Zhejiang University（浙江大学集成电路学院）；Shanghai Innovation Institute（上海创新研究院）；School of Electrical and Electronic Engineering (EEE), Nanyang Technological University（南洋理工大学电子与电气工程学院）；School of Data Science, Fudan University（复旦大学数据科学学院）
InstaDeep（InstaDeep公司）；Digital Research Center of Sfax（斯法克斯数字研究中心）
The Transformer has quickly become the dominant architecture for various pattern recognition tasks due to its capacity for long-range representation. However, transformers are data-hungry models and need large datasets for training. In Handwritten Text Recognition (HTR), collecting a massive amount of labeled data is a complicated and expensive task. In this paper, we propose a lite transformer architecture for full-page multi-script handwriting recognition. The proposed model comes with three advantages: First, to solve the common problem of data scarcity, we propose a lite transformer model …
Handwriting recognition refers to the identification of written characters. Handwriting recognition has become an acute research area in recent years for the ease of access of computer science. In this paper primarily discussed On-line and Off-line handwriting recognition methods for Arabic words which are often used among then across the Middle East and North Africa People. Arabic word online handwriting recognition is a very challenging task due to its cursive nature. Because of the characteristic of the whole body of the Arabic script, namely connectivity between the characters, thereby the…
University of Sfax（斯法克斯大学）；National Engineering School of Sfax（斯法克斯国立工程师学院）；Sifast
Due to the omnipresence of mobile devices, online handwritten scripts have become the most important feeding input to smartphones and tablet devices. To increase online handwriting recognition performance, deeper neural networks have extensively been used. In this context, our paper handles the problem of online handwritten script recognition based on extraction features system and deep approach system for sequences classification. Many solutions have appeared in order to facilitate the recognition of handwriting. Accordingly, we used an existent method and combined with new classifiers in ord…
The ultimate aim of handwriting recognition is to make computers able to read and/or authenticate human written texts, with a performance comparable to or even better than that of humans. Reading means that the computer is given a piece of handwriting and it provides the electronic transcription of that (e.g. in ASCII format). Two types of handwriting: on-line and offline. The most important purpose of off-line handwriting recognition is in protection systems and authentication. Arabic Handwriting scripts are much more complicated in comparison to Latin scripts. This paper introduces a simple …
This paper proposed a method to imitate handwriting style by style transfer. We proposed an neural network model based on conditional generative adversarial networks (cGAN) for handwriting style transfer. This paper improved the loss function on the basis of the GAN. Compared with other handwriting imitation methods, the handwriting style transfer's effect and efficiency have been significantly improved. The experiments showed that the shape of the generated Chinese characters is clear and the analysis of experimental data showed the Generative adversarial networks showed excellent performance…
Techno India College of Technology（Techno India 技术学院）；Jadavpur University（jadavpur 大学）
In the present work, we have used Tesseract 2.01 open source Optical Character Recognition (OCR) Engine under Apache License 2.0 for recognition of handwriting samples of lower case Roman script. Handwritten isolated and free-flow text samples were collected from multiple users. Tesseract is trained to recognize user-specific handwriting samples of both the categories of document pages. On a single user model, the system is trained with 1844 isolated handwritten characters and the performance is tested on 1133 characters, taken form the test set. The overall character-level accuracy of the sys…
Institute of Engineering & Management（工程与管理学院）；Indian Institute of Technology Roorkee（印度理工学院鲁尔基分校）；Indian Statistical Institute（印度统计研究所）
Handwritten word recognition and spotting of low-resource scripts are difficult as sufficient training data is not available and it is often expensive for collecting data of such scripts. This paper presents a novel cross language platform for handwritten word recognition and spotting for such low-resource scripts where training is performed with a sufficiently large dataset of an available script (considered as source script) and testing is done on other scripts (considered as target script). Training with one source script and testing with another script to have a reasonable result is not ea…
Indian Institute of Technology Kharagpur（印度理工学院卡拉格普尔分校）
Handwriting is a skill learned by humans from a very early age. The ability to develop one's own unique handwriting as well as mimic another person's handwriting is a task learned by the brain with practice. This paper deals with this very problem where an intelligent system tries to learn the handwriting of an entity using Generative Adversarial Networks (GANs). We propose a modified architecture of DCGAN (Radford, Metz, and Chintala 2015) to achieve this. We also discuss about applying reinforcement learning techniques to achieve faster learning. Our algorithm hopes to give new insights in t…
Arabic handwriting is a consonantal and cursive writing. The analysis of Arabic script is further complicated due to obligatory dots/strokes that are placed above or below most letters and usually written delayed in order. Due to ambiguities and diversities of writing styles, recognition systems are generally based on a set of possible words called lexicon. When the lexicon is small, recognition accuracy is more important as the recognition time is minimal. On the other hand, recognition speed as well as the accuracy are both critical when handling large lexicons. Arabic is rich in morphology …
Threats to Arabic Handwriting Recognition: Investigating Black-Box Adversarial Attacks on embedded ConvNet models
Systems theory and informatics laboratory（系统理论与信息学实验室）；Moulay Ismail University of Meknes（穆莱·艾息姆大学）；Department of Computer Science（计算机科学系）；EST of Sidi Bennour（西迪·本努尔工程与技术学院）；Chouaib Doukkali University（侯赛因·杜克利大学）；Faculty of Education Sciences（教育科学学院）；University Mohammed V（穆莱·维大学）；Laboratory of Computer Science and Applications（计算机科学与应用实验室）
Generative Adversarial Networks (GAN) receive great attentions recently due to its excellent performance in image generation, transformation, and super-resolution. However, GAN has rarely been studied and trained for classification, leading that the generated images may not be appropriate for classification. In this paper, we propose a novel Generative Adversarial Classifier (GAC) particularly for low-resolution Handwriting Character Recognition. Specifically, involving additionally a classifier in the training process of normal GANs, GAC is calibrated for learning suitable structures and rest…
Brigham Young University（杨百翰大学）；Adobe Research（奥多比研究院）
Training state-of-the-art offline handwriting recognition (HWR) models requires large labeled datasets, but unfortunately such datasets are not available in all languages and domains due to the high cost of manual labeling.We address this problem by showing how high resource languages can be leveraged to help train models for low resource languages.We propose a transfer learning methodology where we adapt HWR models trained on a source language to a target language that uses the same writing script.This methodology only requires labeled data in the source language, unlabeled data in the target…
Université PSL（巴黎文理研究大学）；University of Maryland（马里兰大学）
The application of handwritten text recognition to historical works is highly dependant on accurate text line retrieval. A number of systems utilizing a robust baseline detection paradigm have emerged recently but the advancement of layout analysis methods for challenging scripts is held back by the lack of well-established datasets including works in non-Latin scripts. We present a dataset of 400 annotated document images from different domains and time periods. A short elaboration on the particular challenges posed by handwriting in Arabic script for layout analysis and subsequent processing…
Generative adversarial networks (GANs) have proven hugely successful in variety of applications of image processing. However, generative adversarial networks for handwriting is relatively rare somehow because of difficulty of handling sequential handwriting data by Convolutional Neural Network (CNN). In this paper, we propose a handwriting generative adversarial network framework (HWGANs) for synthesizing handwritten stroke data. The main features of the new framework include: (i) A discriminator consists of an integrated CNN-Long-Short-Term- Memory (LSTM) based feature extraction with Path Si…
Prince Sultan University（苏丹亲王大学）；Universiti Teknologi Malaysia（马来西亚理工大学）
Online Arabic cursive character recognition is still a big challenge due to the existing complexities including Arabic cursive script styles, writing speed, writer mood and so forth. Due to these unavoidable constraints, the accuracy of online Arabic character's recognition is still low and retain space for improvement. In this research, an enhanced method of detecting the desired critical points from vertical and horizontal direction-length of handwriting stroke features of online Arabic script recognition is proposed. Each extracted stroke feature divides every isolated character into some m…
National Institute of Technology Karnataka（卡纳塔克国家理工学院）
We describe a method for classification of handwritten Kannada characters using Hidden Markov Models (HMMs). Kannada script is agglutinative, where simple shapes are concatenated horizontally to form a character. This results in a large number of characters making the task of classification difficult. Character segmentation plays a significant role in reducing the number of classes. Explicit segmentation techniques suffer when overlapping shapes are present, which is common in the case of handwritten text. We use HMMs to take advantage of the agglutinative nature of Kannada script, which allow…
Techno India College of Technology（Techno India 技术学院）；Jadavpur University（Jadavpur 大学）
The objective of the paper is to recognize handwritten samples of lower case Roman script using Tesseract open source Optical Character Recognition (OCR) engine under Apache License 2.0. Handwritten data samples containing isolated and free-flow text were collected from different users. Tesseract is trained with user-specific data samples of both the categories of document pages to generate separate user-models representing a unique language-set. Each such language-set recognizes isolated and free-flow handwritten test samples collected from the designated user. On a three user model, the syst…
South China University of Technology（华南理工大学）；Peng Cheng Laboratory（鹏城实验室）
Large amounts of labeled data are urgently required for the training of robust text recognizers. However, collecting handwriting data of diverse styles, along with an immense lexicon, is considerably expensive. Although data synthesis is a promising way to relieve data hunger, two key issues of handwriting synthesis, namely, style representation and content embedding, remain unsolved. To this end, we propose a novel method that can synthesize parameterized and controllable handwriting Styles for arbitrary-Length and Out-of-vocabulary text based on a Generative Adversarial Network (GAN), termed…
University of Tabriz（大不里士大学）；Amirkabir University of Technology（阿米尔卡比尔理工大学）；University of Tartu（塔尔图大学）
Digit, letter and word recognition for a particular script has various applications in todays commercial contexts. Nevertheless, only a limited number of relevant studies have dealt with Persian scripts. In this paper, deep neural networks are utilized through various DensNet architectures, as well as the Xception, are adopted, modified and further boosted through data augmentation and test time augmentation, in order to come up with an optical character recognition accounting for the particularities of the Persian language and the corresponding handwritings. Taking advantage of dividing the d…
SpiS-GAN: Spiral-Modulated Handwriting Synthesis with Star Operation
University of Information Technology（信息技术大学）；Vietnam National University（越南国立大学）；AJ Technologies（AJ科技公司）；RIKEN Center for Advanced Intelligence Project（理化学研究所先进智能项目中心）
King Saud Bin Abdul Aziz University for Health Sciences（沙特·本·阿卜杜勒阿齐兹国王健康科学大学）；Hazara University（哈扎拉大学）；Higher Education Department, KPK（开伯尔-普赫图赫瓦省高等教育部）；COMSATS Institute of Information Technology（COMSATS 信息技术学院）
The recognition of cursive script is regarded as a subtle task in optical character recognition due to its varied representation. Every cursive script has different nature and associated challenges. As Urdu is one of cursive language that is derived from Arabic script, thats why it nearly shares the same challenges and difficulties even more harder. We can categorized Urdu and Arabic language on basis of its script they use. Urdu is mostly written in Nastaliq style whereas, Arabic follows Naskh style of writing. This paper presents new and comprehensive Urdu handwritten offline database name U…
Actually, the ability to identify the documents authors provides more chances for using these documents for various purposes. In this paper, we present a new effective biometric writer identification system from online handwriting. The system consists of the preprocessing and the segmentation of online handwriting into a sequence of Beta strokes in a first step. Then, from each stroke, we extract a set of static and dynamic features from new proposed model that we called Extended Beta-Elliptic model and from the Fuzzy Elementary Perceptual Codes. Next, all the segments which are composed of N …
Uppsala University（乌普萨拉大学）；Centre for Image Analysis（图像分析中心）
One of the factors limiting the performance of handwritten text recognition (HTR) for stenography is the small amount of annotated training data. To alleviate the problem of data scarcity, modern HTR methods often employ data augmentation. However, due to specifics of the stenographic script, such settings may not be directly applicable for stenography recognition. In this work, we study 22 classical augmentation techniques, most of which are commonly used for HTR of other scripts, such as Latin handwriting. Through extensive experiments, we identify a group of augmentations, including for exa…
Extracting Handwritten text is one of the most important components of digitizing information and making it available for large scale setting. Handwriting Optical Character Reader (OCR) is a research problem in computer vision and natural language processing computing, and a lot of work has been done for English, but unfortunately, very little work has been done for low resourced languages such as Urdu. Urdu language script is very difficult because of its cursive nature and change of shape of characters based on it's relative position, therefore, a need arises to propose a model which can und…
In handwritten character recognition, benchmark database plays an important role in evaluating the performance of various algorithms and the results obtained by various researchers. In Devnagari script, there is lack of such official benchmark. This paper focuses on the generation of offline benchmark database for Devnagari handwritten numerals and characters. The present work generated 5137 and 20305 isolated samples for numeral and character database, respectively, from 750 writers of all ages, sex, education, and profession. The offline sample images are stored in TIFF image format as it oc…
Vidyalankar Institute of Technology（维迪亚兰卡理工学院）；Birla Institute of Technology and Science（比尔拉理工学院）
The working of neural networks following the adversarial philosophy to create a generative model is a fascinating field. Multiple papers have already explored the architectural aspect and proposed systems with potentially good results however, very few papers are available which implement it on a real-world example. Traditionally, people use the famous MNIST dataset as a Hello, World! example for implementing Generative Adversarial Networks (GAN). Instead of going the standard route of using handwritten digits, this paper uses the Devanagari script which has a more complex structure. As there …
BeHGAN: Bengali Handwritten Word Generation from Plain Text Using Generative Adversarial Networks
Department of CSE, Ahsanullah University of Science and Technology, Bangladesh（阿斯南大学科学与技术学院）；Enosis Solutions, Bangladesh（恩osis解决方案有限公司）；Smart Technologies BD Ltd.（智能技术BD有限公司）；Therap BD Ltd.（疗法BD有限公司）；Department of CSE, BRAC University, Bangladesh（布拉克大学计算机科学与工程学院）
State-of-the-art offline handwriting text recognition systems tend to use neural networks and therefore require a large amount of annotated data to be trained. In order to partially satisfy this requirement, we propose a system based on Generative Adversarial Networks (GAN) to produce synthetic images of handwritten words. We use bidirectional LSTM recurrent layers to get an embedding of the word to be rendered, and we feed it to the generator network. We also modify the standard GAN by adding an auxiliary network for text recognition. The system is then trained with a balanced combination of …
As a new way of human-computer interaction, inertial sensor based in-air handwriting can provide a natural and unconstrained interaction to express more complex and richer information in 3D space. However, most of the existing in-air handwriting work is mainly focused on handwritten character recognition, which makes these work suffer from poor readability of inertial signal and lack of labeled samples. To address these two problems, we use unsupervised domain adaptation method to reconstruct the trajectory of inertial signal and generate inertial samples using online handwritten trajectories.…
Many studies on (Offline) Handwritten Text Recognition (HTR) systems have focused on building state-of-the-art models for line recognition on small corpora. However, adding HTR capability to a large scale multilingual OCR system poses new challenges. This paper addresses three problems in building such systems: data, efficiency, and integration. Firstly, one of the biggest challenges is obtaining sufficient amounts of high quality training data. We address the problem by using online handwriting data collected for a large scale production online handwriting recognition system. We describe our …
Techno India College of Technology（Techno India 技术学院）；Jadavpur University（贾达沃普尔大学）；Hitachi Limited（日立有限公司）
Objective of the current work is to develop an Optical Character Recognition (OCR) engine for information Just In Time (iJIT) system that can be used for recognition of handwritten textual annotations of lower case Roman script. Tesseract open source OCR engine under Apache License 2.0 is used to develop user-specific handwriting recognition models, viz., the language sets, for the said system, where each user is identified by a unique identification tag associated with the digital pen. To generate the language set for any user, Tesseract is trained with labeled handwritten data samples of iso…
University of Kurdistan Hewler（休勒库尔德斯坦大学）；Singidunum University（辛吉杜努姆大学）；Torrens University（托伦斯大学）；Yonsei University（延世大学）；Ramco Institute of Technology（拉姆科理工学院）；Guru Nanak Dev University（古鲁那纳克德夫大学）；Tishk International University（提什克国际大学）
Handwriting recognition is one of the active and challenging areas of research in the field of image processing and pattern recognition. It has many applications that include: a reading aid for visual impairment, automated reading and processing for bank checks, making any handwritten document searchable, and converting them into structural text form, etc. Moreover, high accuracy rates have been recorded by handwriting recognition systems for English, Chinese Arabic, Persian, and many other languages. Yet there is no such system available for offline Kurdish handwriting recognition. In this pa…
We introduce a new dataset for offline Handwritten Text Recognition (HTR) from images of Bangla scripts comprising words, lines, and document-level annotations. The BN-HTRd dataset is based on the BBC Bangla News corpus, meant to act as ground truth texts. These texts were subsequently used to generate the annotations that were filled out by people with their handwriting. Our dataset includes 788 images of handwritten pages produced by approximately 150 different writers. It can be adopted as a basis for various handwriting classification tasks such as end-to-end document recognition, word-spo…
National Engineering School of Sfax, University of Sfax（斯法克斯国立工程学院，斯法克斯大学）
In this paper, we propose an automatic analysis system for the Arabic handwriting postal addresses recognition, by using the beta elliptical model. Our system is divided into different steps: analysis, pre-processing and classification. The first operation is the filtering of image. In the second, we remove the border print, stamps and graphics. After locating the address on the envelope, the address segmentation allows the extraction of postal code and city name separately. The pre-processing system and the modeling approach are based on two basic steps. The first step is the extraction of th…
University of Würzburg（维尔茨堡大学）；CIS, LMU Munich（慕尼黑大学信息与语言处理中心）
This paper deals with the task of practical and open source Handwritten Text Recognition (HTR) on German medieval manuscripts. We report on our efforts to construct mixed recognition models which can be applied out-of-the-box without any further document-specific training but also serve as a starting point for finetuning by training a new model on a few pages of transcribed text (ground truth). To train the mixed models we collected a corpus of 35 manuscripts and ca. 12.5k text lines for two widely used handwriting styles, Gothic and Bastarda cursives. Evaluating the mixed models out-of-the-bo…
