Hybrid CNN-ViT Framework for Motion-Blurred Scene Text Restoration
Department of Electrical Engineering, University of Engineering & Technology, New Campus, Lahore, Pakistan（电子工程系，工程科技大学，新校区，拉合尔，巴基斯坦）；Department of Electrical, Electronic, and Future Technologies, Sheffield Hallam University, Sheffield S1 1WB, UK（电子、电气与未来技术系，谢菲尔德哈姆大学，谢菲尔德 S1 1WB，英国）
Umar Rashid, Muhammad Arslan Arshad, Ghulam Ahmad, Muhammad Zeeshan Anjum, Rizwan Khan, Muhammad Akmal
Motion blur in scene text images severely impairs readability and hinders the reliability of computer vision tasks, including autonomous driving, document digitization, and visual information retrieval. Conventional deblurring approaches are often inadequate in handling spatially varying blur and typically fall short in modeling the long-range dependencies necessary for restoring textual clarity. To overcome these limitations, we introduce a hybrid deep learning framework that combines convolutional neural networks (CNNs) with vision transformers (ViTs), thereby leveraging both local feature extraction and global contextual reasoning. The architecture employs a CNN-based encoder-decoder to preserve structural details, while a transformer module enhances global awareness through self-attention. Training is conducted on a curated dataset derived from TextOCR, where sharp scene-text samples are paired with synthetically blurred versions generated using realistic motion-blur kernels of multiple sizes and orientations. Model optimization is guided by a composite loss that incorporates mean absolute error (MAE), squared error (MSE), perceptual similarity, and structural similarity (SSIM). Quantitative evaluations show that the proposed method attains 32.20 dB in PSNR and 0.934 in SSIM, while remaining lightweight with 2.83 million parameters and an average inference time of 61 ms. These results highlight the effectiveness and computational efficiency of the CNN-ViT hybrid design, establishing its practicality for real-world motion-blurred scene-text restoration.
We propose a novel framework to generate clean video frames from a single motion-blurred image. While a broad range of literature focuses on recovering a single image from a blurred image, in this work, we tackle a more challenging task i.e. video restoration from a blurred image. We formulate video restoration from a single blurred image as an inverse problem by setting clean image sequence and their respective motion as latent factors, and the blurred image as an observation. Our framework is based on an encoder-decoder structure with spatial transformer network modules to restore a video se…
Beijing University of Technology（北京工业大学）；Beijing Key Laboratory of Computational Intelligence and Intelligent System（计算智能与智能系统北京市重点实验室）；The Hong Kong Polytechnic University（香港理工大学）
Dynamic scene deblurring is a challenging problem in computer vision. It is difficult to accurately estimate the spatially varying blur kernel by traditional methods. Data-driven-based methods usually employ kernel-free end-to-end mapping schemes, which are apt to overlook the kernel estimation. To address this issue, we propose a blur-attention module to dynamically capture the spatially varying features of non-uniform blurred images. The module consists of a DenseBlock unit and a spatial attention unit with multi-pooling feature fusion, which can effectively extract complex spatially varying…
School of Informatics（信息学院）；Xiamen University（厦门大学）
Low-quality face image restoration is a popular research direction in today's computer vision field. It can be used as a pre-work for tasks such as face detection and face recognition. At present, there is a lot of work to solve the problem of low-quality faces under various environmental conditions. This paper mainly focuses on the restoration of motion-blurred faces. In increasingly abundant mobile scenes, the fast recovery of motion-blurred faces can bring highly effective speed improvements in tasks such as face matching. In order to achieve this goal, a deblurring method for motion-blurre…
RetinaRegen: A Hybrid Model for Readability and Detail Restoration in Fundus Images
Sichuan University（四川大学）；International School,Beijing University of Posts and Telecommunications（北京邮电大学国际学院）；School of Computer Science and Technology, University of Chinese Academy of Sciences（中国科学院大学计算机科学与技术学院）
Chongqing University of Posts and Telecommunications（重庆邮电大学）；Chongqing University（重庆大学）
Most existing deep-learning-based single image dynamic scene blind deblurring (SIDSBD) methods usually design deep networks to directly remove the spatially-variant motion blurs from one inputted motion blurred image, without blur kernels estimation. In this paper, inspired by the Projective Motion Path Blur (PMPB) model and deformable convolution, we propose a novel constrained deformable convolutional network (CDCN) for efficient single image dynamic scene blind deblurring, which simultaneously achieves accurate spatially-variant motion blur kernels estimation and the high-quality image rest…
Camera motion deblurring is an important low-level vision task for achieving better imaging quality. When a scene has outliers such as saturated pixels, the captured blurred image becomes more difficult to restore. In this paper, we propose a novel method to handle camera motion blur with outliers. We first propose an edge-aware scale-recurrent network (EASRN) to conduct deblurring. EASRN has a separate deblurring module that removes blur at multiple scales and an upsampling module that fuses different input scales. Then a salient edge detection network is proposed to supervise the training pr…
Sharing Key Semantics in Transformer Makes Efficient Image Restoration
ETH Zürich（苏黎世联邦理工学院）；INSAIT, Sofia University（保加利亚索菲亚大学INSAIT研究院）；University of Pisa（比萨大学）；University of Trento（特伦托大学）；Meta Reality Labs（元宇宙现实实验室）；Peking University（北京大学）；University of Modena and Reggio Emilia（摩德纳-雷焦艾米利亚大学）；University of California, Merced（加州大学默塞德分校）
How Powerful Potential of Attention on Image Restoration?
Sun Yat-sen University（中山大学）；The Hong Kong Polytechnic University（香港理工大学）；Nanjing University of Science and Technology（南京理工大学）；National University of Singapore（新加坡国立大学）；Dalian University of Technology（大连理工大学）
Unified Restoration-Perception Learning: Maritime Infrared-Visible Image Fusion and Segmentation
Laboratory of Underwater Acoustic Communication and Marine Information Technology, Ministry of Education, Xiamen University（厦门大学水声通信与海洋信息技术教育部重点实验室）；PAMI Research Group, Department of Computer and Information Science, University of Macau（澳门大学计算机与信息科学系PAMI研究组）；School of Cyber Science and Technology, Shenzhen Campus of Sun Yat-sen University（中山大学深圳校区网络空间安全学院）
Prompt-CAM: Making Vision Transformers Interpretable for Fine-Grained Analysis
The Ohio State University（俄亥俄州立大学）；University of Tsukuba（筑波大学）；Virginia Tech（弗吉尼亚理工大学）；Princeton University（普林斯顿大学）；Rensselaer Polytechnic Institute（伦斯勒理工学院）
Bahcesehir University（巴赫切谢希尔大学）；American University of Malta（马耳他美国大学）
Medical imaging plays a significant role in detecting and treating various diseases. However, these images often happen to be of too poor quality, leading to decreased efficiency, extra expenses, and even incorrect diagnoses. Therefore, we propose a retinal image enhancement method using a vision transformer and convolutional neural network. It builds a cycle-consistent generative adversarial network that relies on unpaired datasets. It consists of two generators that translate images from one domain to another (e.g., low- to high-quality and vice versa), playing an adversarial game with two d…
National Tsing Hua University（国立清华大学）；National Chengchi University（国立政治大学）；Qualcomm Technologies, Inc.（高通技术公司）；National Yang Ming Chiao Tung University（国立阳明交通大学）
Image motion blur results from a combination of object motions and camera shakes, and such blurring effect is generally directional and non-uniform. Previous research attempted to solve non-uniform blurs using self-recurrent multiscale, multi-patch, or multi-temporal architectures with self-attention to obtain decent results. However, using self-recurrent frameworks typically lead to a longer inference time, while inter-pixel or inter-channel self-attention may cause excessive memory usage. This paper proposes a Blur-aware Attention Network (BANet), that accomplishes accurate and efficient deb…
Universidad de la República（乌拉圭共和国大学）；Universitat Pompeu Fabra（庞培法布拉大学）
Motion blur estimation remains an important task for scene analysis and image restoration. In recent years, the removal of motion blur in photographs has seen impressive progress in the hands of deep learning-based methods, trained to map directly from blurry to sharp images. Characterization of the motion blur, on the other hand, has received less attention, and progress in model-based methods for deblurring lags behind that of data-driven end-to-end approaches. In this work we revisit the problem of characterizing dense, non-uniform motion blur in a single image and propose a general non-par…
Dalian University of Technology（大连理工大学）；Stony Brook University（石溪大学）
Blind image deblurring algorithms have been improving steadily in the past years. Most state-of-the-art algorithms, however, still cannot perform perfectly in challenging cases, especially in large blur setting. In this paper, we focus on how to estimate a good kernel estimate from a single blurred image based on the image structure. We found that image details caused by blurring could adversely affect the kernel estimation, especially when the blur kernel is large. One effective way to eliminate these details is to apply image denoising model based on the Total Variation (TV). First, we devel…
Guangdong University of Technology（广东工业大学）；South China University of Technology（华南理工大学）；Sun Yat-sen University（中山大学）；Nanjing University of Science and Technology（南京理工大学）
Real-world image denoising is a practical image restoration problem that aims to obtain clean images from in-the-wild noisy inputs. Recently, the Vision Transformer (ViT) has exhibited a strong ability to capture long-range dependencies, and many researchers have attempted to apply the ViT to image denoising tasks. However, a real-world image is an isolated frame that makes the ViT build long-range dependencies based on the internal patches, which divides images into patches, disarranges noise patterns and damages gradient continuity. In this article, we propose to resolve this issue by using …
Southern University of Science and Technology（南方科技大学）；University of Technology Sydney（悉尼科技大学）；Tomey Corporation（Tomey公司）；Inception Institute of Artificial Intelligence（起源人工智能研究院）；Baidu（百度）；Chinese Academy of Sciences（中国科学院）
Corneal endothelial cell segmentation plays a vital role inquantifying clinical indicators such as cell density, coefficient of variation,and hexagonality. However, the corneal endothelium's uneven reflectionand the subject's tremor and movement cause blurred cell edges in theimage, which is difficult to segment, and need more details and contextinformation to release this problem. Due to the limited receptive field oflocal convolution and continuous downsampling, the existing deep learn-ing segmentation methods cannot make full use of global context andmiss many details. This paper proposes a…
Department of Artificial Intelligence（人工智能系）；Kyungpook National University（庆尚国立大学）；Department of Electrical and Computer Engineering（电气电子工程系）；Queen’s University（皇后大学）；Department of Information and Communications Engineering（信息与通信工程系）；Pukyong National University（浦项国立大学）
Image super-resolution(SR) is fundamental to many vision system-from surveillance and autonomy to document analysis and retail analytics-because recovering high-frequency details, especially scene-text, enables reliable downstream perception. Scene-text, i.e., text embedded in natural images such as signs, product labels, and storefronts, often carries the most actionable information; when characters are blurred or hallucinated, optical character recognition(OCR) and subsequent decisions fail even if the rest of the image appears sharp. Yet previous SR research has often been tuned to distorti…
Motion-Aware Adaptive Pixel Pruning for Efficient Local Motion Deblurring
Harbin Institute of Technology \& City University of Hong Kong；School of Computer Science；Technology, Harbin Institute of Technology；Tianjin University \& Low-Altitude Intelligence Lab, Xiong'an National Innovation Center Technology Co., Ltd；College of Intelligence；Computing, Tianjin University
SSMB: Self-Supervised Local Feature Detection under Motion Blur
University of Zaragoza（萨拉戈萨大学）；University of Palermo（巴勒莫大学）；The Chinese University of Hong Kong（香港中文大学）；Tohoku University（东北大学）；Central South University（中南大学）；Texas A&M University（德克萨斯农工大学）；Nanjing University of Aeronautics and Astronautics（南京航空航天大学）；The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））
