A Robust Deep Learning Framework for Bangla License Plate Recognition Using YOLO and Vision-Language OCR
Department of Electrical（电气工程系）；Electronic Engineering Islamic University of Technology Gazipur, Bangladesh（电子工程伊斯兰科技大学加兹布尔，孟加拉国）；Department of Mechanical Engineering Bangladesh University Of Engineering And Technology Dhaka, Bangladesh（机械工程孟加拉国工程与技术大学达卡，孟加拉国）
自动车牌识别（ALPR）系统是智能交通管理系统的关键组成部分。然而，由于复杂的字符方案和不均匀的布局，孟加拉车牌检测仍然具有挑战性。本文提出了一种鲁棒的孟加拉车牌识别系统，该系统将基于深度学习的车牌定位目标检测模型与用于文本提取的光学字符识别相结合。比较了多种目标检测架构，包括U-Net和几种YOLO（You Only Look Once）变体，用于车牌定位。本研究提出了一种基于YOLOv8架构的新型两阶段自适应训练策略，以提高定位性能。所提出的方法优于现有模型，达到了97.83%的准确率和91.3%的交并比（IoU）。文本识别问题被表述为基于视觉编码器-解码器架构的序列生成问题，并评估了编码器-解码器的组合。结果表明，ViT + BanglaBERT模型在字符级别上取得了更好的结果，字符错误率为0.1323，词错误率为0.1068。所提出的系统在为此研究目的整理的外部数据集上进行测试时也表现出一致的性能。与训练样本相比，该数据集提供了完全不同的环境和光照条件，表明了所提出框架的鲁棒性。总体而言，我们提出的系统为孟加拉车牌识别提供了鲁棒且可靠的解决方案，并在各种真实场景中有效运行，包括光照、噪声和车牌样式的变化。这些优势使其非常适合部署在智能交通应用中，如自动执法和访问控制。
英文摘要：
An Automatic License Plate Recognition (ALPR) system constitutes a crucial element in an intelligent traffic management system. However, the detection of Bangla license plates remains challenging because of the complicated character scheme and uneven layouts. This paper presents a robust Bangla License Plate Recognition system that integrates a deep learning-based object detection model for license plate localization with Optical Character Recognition for text extraction. Multiple object detection architectures, including U-Net and several YOLO (You Only Look Once) variants, are compared for license plate localization. This study proposes a novel two-stage adaptive training strategy built upon the YOLOv8 architecture to improve localization performance. The proposed approach outperforms the established models, achieving an accuracy of 97.83% and an Intersection over Union (IoU) of 91.3%. The text recognition problem is phrased as a sequence generation problem with a VisionEncoderDecoder architecture, with a combination of encoder-decoders evaluated. It was demonstrated that the ViT + BanglaBERT model gives better results at the character level, with a Character Error Rate of 0.1323 and Word Error Rate of 0.1068. The proposed system also shows a consistent performance when tested on an external dataset that has been curated for this study purpose. The dataset offers completely different environment and lighting conditions compared to the training sample, indicating the robustness of the proposed framework. Overall, our proposed system provides a robust and reliable solution for Bangla license plate recognition and performs effectively across diverse real-world scenarios, including variations in lighting, noise, and plate styles. These strengths make it well suited for deployment in intelligent transportation applications such as automated law enforcement and access control.
Shahjalal University of Science & Technology（沙赫贾拉勒科技大学）
Developing a Bangla OCR requires bunch of algorithm and methods. There were many effort went on for developing a Bangla OCR. But all of them failed to provide an error free Bangla OCR. Each of them has some lacking. We discussed about the problem scope of currently existing Bangla OCR's. In this paper, we present the basic steps required for developing a Bangla OCR and a complete workflow for development of a Bangla OCR with mentioning all the possible algorithms required.
Color Independent Word Segmentation From Transcribed Bangla Passages
Department of Computer Science；Engineering Khulna University of Engineering \& Technology Khulna-9203, Bangladesh；Engineering Khulna University of Engineering \& Technology Khulna-9203, Bangladesh Email: 1 2 3
MCKV Institute of Engineering（MCKV工程学院）；Jadavpur University（贾达普大学）
The work presented here involves the design of a Multi Layer Perceptron (MLP) based pattern classifier for recognition of handwritten Bangla digits using a 76 element feature vector. Bangla is the second most popular script and language in the Indian subcontinent and the fifth most popular language in the world. The feature set developed for representing handwritten Bangla numerals here includes 24 shadow features, 16 centroid features and 36 longest-run features. On experimentation with a database of 6000 samples, the technique yields an average recognition rate of 96.67% evaluated after thre…
Extraction and recognition of Bangla text from video frame images is challenging due to complex color background, low-resolution etc. In this paper, we propose an algorithm for extraction and recognition of Bangla text form such video frames with complex background. Here, a two-step approach has been proposed. First, the text line is segmented into words using information based on line contours. First order gradient value of the text blocks are used to find the word gap. Next, a local binarization technique is applied on each word and text line is reconstructed using those words. Secondly, thi…
A novel approach for recognition of handwritten compound Bangla characters, along with the Basic characters of Bangla alphabet, is presented here. Compared to English like Roman script, one of the major stumbling blocks in Optical Character Recognition (OCR) of handwritten Bangla script is the large number of complex shaped character classes of Bangla alphabet. In addition to 50 basic character classes, there are nearly 160 complex shaped compound character classes in Bangla alphabet. Dealing with such a large varieties of handwritten characters with a suitably designed feature set is a challe…
Open Vocabulary Word Recognition From Transcribed Bangla Texts
Department of Computer Science；Engineering Khulna University of Engineering \& Technology Khulna-9203, Bangladesh；Engineering Khulna University of Engineering \& Technology Khulna-9203, Bangladesh Email: 1 3
Techno India College of Technology（泰克诺印度技术学院）；Jadavpur University（贾达普大学）
The objective of the paper is to recognize handwritten samples of basic Bangla characters using Tesseract open source Optical Character Recognition (OCR) engine under Apache License 2.0. Handwritten data samples containing isolated Bangla basic characters and digits were collected from different users. Tesseract is trained with user-specific data samples of document pages to generate separate user-models representing a unique language-set. Each such language-set recognizes isolated basic Bangla handwritten test samples collected from the designated users. On a three user model, the system is t…
Automated Wicket-Taking Delivery Segmentation and Trajectory-Based Dismissal-Zone Analysis in Cricket Videos Using OCR-Guided YOLOv8
Department of Computer Science and Engineering, University of Rajshahi, Rajshahi, Bangladesh（拉贾沙希大学计算机科学与工程系，孟加拉国拉贾沙希）；Marshall University, Huntington, WV, USA（马歇尔大学，美国西弗吉尼亚州亨廷顿）
MCKV Institute of Engineering（MCKV工程学院）；Jadavpur University（贾达普大学）
The work presented here involves the design of a Multi Layer Perceptron (MLP) based classifier for recognition of handwritten Bangla alphabet using a 76 element feature set Bangla is the second most popular script and language in the Indian subcontinent and the fifth most popular language in the world. The feature set developed for representing handwritten characters of Bangla alphabet includes 24 shadow features, 16 centroid features and 36 longest-run features. Recognition performances of the MLP designed to work with this feature set are experimentally observed as 86.46% and 75.05% on the s…
Hajee Mohammad Danesh Science and Technology University（哈吉·穆罕默德·达内什科技大学）
Bangla Handwritten Digit recognition is a significant step forward in the development of Bangla OCR. However, intricate shape, structural likeness and distinctive composition style of Bangla digits makes it relatively challenging to distinguish. Thus, in this paper, we benchmarked four rigorous classifiers to recognize Bangla Handwritten Digit: K-Nearest Neighbor (KNN), Support Vector Machine (SVM), Random Forest (RF), and Gradient-Boosted Decision Trees (GBDT) based on three handcrafted feature extraction techniques: Histogram of Oriented Gradients (HOG), Local Binary Pattern (LBP), and Gabor…
Jadavpur University（jadavpur大学）；Indian Institute of Engineering Science and Technology, Shibpur（印度工程科学技术学院（希布普尔））
Segmentation of handwritten document images into text lines and words is one of the most significant and challenging tasks in the development of a complete Optical Character Recognition (OCR) system. This paper addresses the automatic segmentation of text words directly from unconstrained Bangla handwritten document images. The popular Distance transform (DT) algorithm is applied for locating the outer boundary of the word images. This technique is free from generating the over-segmented words. A simple post-processing procedure is applied to isolate the under-segmented word images, if any. Th…
Society for Natural Language Technology Research（自然语言技术研究协会）；Techno India University（Techno印度大学）
This paper presents a printed Bengali and English text OCR system developed by us using a single hidden BLSTM-CTC architecture having 128 units. Here, we did not use any peephole connection and dropout in the BLSTM, which helped us in getting better accuracy. This architecture was trained by 47,720 text lines that include English words also. When tested over 20 different Bengali fonts, it has produced character level accuracy of 99.32% and word level accuracy of 96.65%. A good Indic multi script OCR system is also developed by Google. It sometimes recognizes a character of Bengali into the sam…
The task of locating and classifying different types of vehicles has become a vital element in numerous applications of automation and intelligent systems ranging from traffic surveillance to vehicle identification and many more. In recent times, Deep Learning models have been dominating the field of vehicle detection. Yet, Bangladeshi vehicle detection has remained a relatively unexplored area. One of the main goals of vehicle detection is its real-time application, where `You Only Look Once' (YOLO) models have proven to be the most effective architecture. In this work, intending to find the …
Bangladesh University of Business & Technology（孟加拉商业与技术大学）；King Abdulaziz University（阿卜杜勒阿齐兹国王大学）
Optical character recognition (OCR) is a process of converting analogue documents into digital using document images. Currently, many commercial and non-commercial OCR systems exist for both handwritten and printed copies for different languages. Despite this, very few works are available in case of recognising Bengali words. Among them, most of the works focused on OCR of printed Bengali characters. This paper introduces an end-to-end OCR system for Bengali language. The proposed architecture implements an end to end strategy that recognises handwritten Bengali words from handwritten word ima…
Chiang Mai University（清迈大学）；Vidyasirimedhi Institute of Science & Technology（威迪雅西里梅迪科学技术研究所）；Medensy Co., Ltd.（Medensy有限公司）；Mahidol University（玛希隆大学）
In light of the COVID-19 pandemic, patients were required to manually input their daily oxygen saturation (SpO2) and pulse rate (PR) values into a health monitoring system-unfortunately, such a process trend to be an error in typing. Several studies attempted to detect the physiological value from the captured image using optical character recognition (OCR). However, the technology has limited availability with high cost. Thus, this study aimed to propose a novel framework called PACMAN (Pandemic Accelerated Human-Machine Collaboration) with a low-resource deep learning-based computer vision. …
Islamic University of Technology（伊斯兰科技大学）；Pioneer Alpha Ltd.（Pioneer Alpha有限公司）
Automatic License Plate Recognition systems aim to provide a solution for detecting, localizing, and recognizing license plate characters from vehicles appearing in video frames. However, deploying such systems in the real world requires real-time performance in low-resource environments. In our paper, we propose a two-stage detection pipeline paired with Vision API that provides real-time inference speed along with consistently accurate detection and recognition performance. We used a haar-cascade classifier as a filter on top of our backbone MobileNet SSDv2 detection model. This reduces infe…
Unfolding the Past: A Comprehensive Deep Learning Approach to Analyzing Incunabula Pages
Department of Human-Centered Artificial Intelligence（人机协同人工智能系）；Institute of Applied Computer Science（应用计算机科学研究所）；Faculty of Physics, Astronomy and Applied Computer Science（物理、天文学与应用计算机科学学院）
Military Institute of Science and Technology（军事科学技术研究院）
The development of the Automatic License Plate Recognition (ALPR) system has received much attention for the English license plate. However, despite being the sixth largest population around the world, no significant progress can be tracked in the Bengali language countries or states for the ALPR system addressing their more alarming traffic management with inadequate road-safety measures. This paper reports a computationally efficient and reasonably accurate Automatic License Plate Recognition (ALPR) system for Bengali characters with a new end-to-end DNN model that we call Bengali License Pl…
We introduce Chinese Text in the Wild, a very large dataset of Chinese text in street view images. While optical character recognition (OCR) in document images is well studied and many commercial tools are available, detection and recognition of text in natural images is still a challenging problem, especially for more complicated character sets such as Chinese text. Lack of training data has always been a problem, especially for deep learning methods which require massive training data. In this paper we provide details of a newly created dataset of Chinese text with about 1 million Chinese ch…
Military Institute of Science & Technology (MIST)（军事科学技术研究院）
Computer vision coupled with Deep Learning (DL) techniques bring out a substantial prospect in the field of traffic control, monitoring and law enforcing activities. This paper presents a YOLOv4 object detection model in which the Convolutional Neural Network (CNN) is trained and tuned for detecting the license plate of the vehicles of Bangladesh and recognizing characters using tesseract from the detected license plates. Here we also present a Graphical User Interface (GUI) based on Tkinter, a python package. The license plate detection model is trained with mean average precision (mAP) of 90…
School of Electrical Sciences, Indian Institute of Technology Bhubaneswar（印度理工学院布巴内斯瓦尔分校电气科学学院）
Feature representation in the form of spatio-spectral decomposition is one of the robust techniques adopted in automatic handwritten character recognition systems. In this regard, we propose a new image representation approach for unconstrained handwritten alphanumeric characters using sparse concept coded Tetrolets. Tetrolets, which does not use fixed dyadic square blocks for spectral decomposition like conventional wavelets, preserve the localized variations in handwritings by adopting tetrominoes those capture the shape geometry. The sparse concept coding of low entropy Tetrolet representat…
Word-level handwritten optical character recognition (OCR) remains a challenge for morphologically rich languages like Bangla. The complexity arises from the existence of a large number of alphabets, the presence of several diacritic forms, and the appearance of complex conjuncts. The difficulty is exacerbated by the fact that some graphemes occur infrequently but remain indispensable, so addressing the class imbalance is required for satisfactory results. This paper addresses this issue by introducing two knowledge distillation methods: Leveraging Isolated Letter Accumulations By Ordering Tea…
Multi-Head Attention based interaction-aware architecture for Bangla Handwritten Character Recognition: Introducing a Primary Dataset
Department of Computer and Communication Engineering, International Islamic University Chittagong（国际伊斯兰大学吉大港分校计算机与通信工程系）；mPower Social Enterprise（mPower社会企业）；Department of Information and Communication Engineering, Noakhali Science and Technology University（诺阿卡利科技大学信息与通信工程系）；Department of CSE, Netrokona University（内特罗科纳大学计算机科学与工程系）；School of Computing Sciences and Computer Engineering, University of Southern Mississippi（南密西西比大学计算科学与计算机工程学院）
iWatchRoad: Scalable Detection and Geospatial Visualization of Potholes for Smart Cities
National Institute of Science Education and Research (NISER), An OCC of Homi Bhabha National Institute（国家科学教育研究所（NISER），霍米·巴哈瓦国家研究所的下属机构）；Silicon University（硅大学）
University of Rajshahi（拉杰沙希大学）；Rajshahi University of Engineering and Technology（拉杰沙希工程技术大学）；Universiti Kebangsaan Malaysia（马来西亚国民大学）；Independent University, Bangladesh（孟加拉独立大学）
Evaluating Vision-Language Models as a Zero-Shot Learning Alternative to You Only Look Once and Optical Character Recognition for Nigerian License Plate Recognition
Ahmadu Bello University（艾哈迈杜·贝罗大学）；Bayero University Kano（卡诺贝罗大学）
iWatchRoadv2: Pothole Detection, Geospatial Mapping, and Intelligent Road Governance
National Institute of Science Education and Research (NISER), An OCC of HBNI（国家科学教育与研究 institute（NISER），HBNI 的一个办事处）；Silicon Institute of Technology（硅技术学院）
