arXiv 2109.049702021-09-10eess.IVcs.CV

View Blind-spot as Inpainting: Self-Supervised Denoising with Mask Guided Residual Convolution

McGill University（麦吉尔大学）；The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））；Shenzhen Institute of Artificial Intelligence and Robotics for Society（深圳人工智能与机器人研究院）

Yuhongze Zhou, Liguang Zhou, Tin Lun Lam, Yangsheng Xu

阅读 PDF ↗

arXiv 页面 ↗HTML 原文 ↗

英文摘要相关论文

英文摘要：

In recent years, self-supervised denoising methods have shown impressive performance, which circumvent painstaking collection procedure of noisy-clean image pairs in supervised denoising methods and boost denoising applicability in real world. One of well-known self-supervised denoising strategies is the blind-spot training scheme. However, a few works attempt to improve blind-spot based self-denoiser in the aspect of network architecture. In this paper, we take an intuitive view of blind-spot strategy and consider its process of using neighbor pixels to predict manipulated pixels as an inpainting process. Therefore, we propose a novel Mask Guided Residual Convolution (MGRConv) into common convolutional neural networks, e.g. U-Net, to promote blind-spot based denoising. Our MGRConv can be regarded as soft partial convolution and find a trade-off among partial convolution, learnable attention maps, and gated convolution. It enables dynamic mask learning with appropriate mask constrain. Different from partial convolution and gated convolution, it provides moderate freedom for network learning. It also avoids leveraging external learnable parameters for mask activation, unlike learnable attention maps. The experiments show that our proposed plug-and-play MGRConv can assist blind-spot based denoising network to reach promising results on both existing single-image based and dataset-based methods.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

阅读原文

阅读 PDF ↗

arXiv 页面 ↗HTML 原文 ↗

本文内容

英文摘要相关论文

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

继续探索

相关论文

面向人工标记去除的具备目标感知判别能力的盲修复

Blind Inpainting with Object-aware Discrimination for Artificial Marker Removal

ZJU-UIUC Institute, Zhejiang University（浙江大学ZJU-UIUC学院）；University of Washington（华盛顿大学）

针对医学图像人工标记干扰AI诊断且现有修复需手动掩码的问题，提出含双分支重建网络与目标感知判别器的盲修复方法，在三类医学数据集上表现优于SOTA方法。

2024-10-31cs.CVcs.LGeess.IV

Blind-Spot Collision Detection System for Commercial Vehicles Using Multi Deep CNN Architecture

Universiti Teknologi PETRONAS（马来西亚石油科技大学）；Université Paris-Est Créteil (UPEC)（巴黎东克雷泰伊大学）；COMSATS University Islamabad, Lahore Campus（伊斯兰堡COMSATS大学拉合尔校区）

Buses and heavy vehicles have more blind spots compared to cars and other road vehicles due to their large sizes. Therefore, accidents caused by these heavy vehicles are more fatal and result in severe injuries to other road users. These possible blind-spot collisions can be identified early using vision-based object detection approaches. Yet, the existing state-of-the-art vision-based object detection models rely heavily on a single feature descriptor for making decisions. In this research, the design of two convolutional neural networks (CNNs) based on high-level feature descriptors and thei…

2022-08-19cs.CVeess.IV

Blind2Unblind：具有可见盲点的自监督图像去噪

Blind2Unblind: Self-Supervised Image Denoising with Visible Blind Spots

Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；University of Chinese Academy of Sciences（中国科学院大学）

针对盲点驱动自监督去噪方法的信息损失问题，提出Blind2Unblind方法，通过全局感知掩码映射器和重新可见损失，实现高效去噪，在合成和真实数据集上性能优越。

2023-05-08cs.CVeess.IV

重新思考基于Transformer的盲点网络用于自监督图像去噪

Rethinking Transformer-Based Blind-Spot Network for Self-Supervised Image Denoising

Junyi Li, Zhilu Zhang, and Wangmeng Zuo（李俊毅、张志鲁和祖旺萌）

本文针对自监督图像去噪中Transformer注意力违反盲点要求的问题，重新设计通道分组注意力与空间掩码注意力，提出TBSN，并通过知识蒸馏提升效率，在真实数据集上取得领先性能。

2025-08-19cs.CVeess.IV

TM-BSN：三角掩码盲 spot 网络用于真实世界自监督图像去噪

TM-BSN: Triangular-Masked Blind-Spot Network for Real-World Self-Supervised Image Denoising

Department of ECE, INMC, Seoul National University（首尔大学电气与计算机工程系、INMC）；IPAI, Seoul National University（首尔大学IPAI）

本文提出TM-BSN网络，通过三角掩码建模真实sRGB噪声的空间相关性，无需下采样即可实现高效自监督图像去噪，实验表明其性能优于现有方法。

2026-04-06cs.CVeess.IV

Blind Image Denoising and Inpainting Using Robust Hadamard Autoencoders

Worcester Polytechnic Institute（伍斯特理工学院）；Colorado State University（科罗拉多州立大学）

In this paper, we demonstrate how deep autoencoders can be generalized to the case of inpainting and denoising, even when no clean training data is available. In particular, we show how neural networks can be trained to perform all of these tasks simultaneously. While, deep autoencoders implemented by way of neural networks have demonstrated potential for denoising and anomaly detection, standard autoencoders have the drawback that they require access to clean data for training. However, recent work in Robust Deep Autoencoders (RDAEs) shows how autoencoders can be trained to eliminate outliers…

2021-01-26cs.CVcs.LGeess.IV

更多相关论文收起更多论文

Efficient Blind-Spot Neural Network Architecture for Image Denoising

CSEM（瑞士电子与微技术中心）

Image denoising is an essential tool in computational photography. Standard denoising techniques, which use deep neural networks at their core, require pairs of clean and noisy images for its training. If we do not possess the clean samples, we can use blind-spot neural network architectures, which estimate the pixel value based on the neighbouring pixels only. These networks thus allow training on noisy images directly, as they by-design avoid trivial solutions. Nowadays, the blind-spot is mostly achieved using shifted convolutions or serialization. We propose a novel fully convolutional netw…

2020-08-25cs.CVcs.LGeess.IV

结合下采样不变性损失与条件盲点网络的自监督图像去噪

Self-supervised Image Denoising with Downsampled Invariance Loss and Conditional Blind-Spot Network

Seoul National University（首尔国立大学）；Gauss Labs Inc.（高斯实验室公司）

提出一种结合下采样不变性损失与条件盲点网络（C-BSN）的自监督图像去噪框架，通过控制网络盲目性并利用随机子采样器去除空间噪声相关性，在真实数据集上达到最先进性能。

2023-07-28cs.CVeess.IV

Speckle2Void: Deep Self-Supervised SAR Despeckling with Blind-Spot Convolutional Neural Networks

Politecnico di Torino（都灵理工大学）

Information extraction from synthetic aperture radar (SAR) images is heavily impaired by speckle noise, hence despeckling is a crucial preliminary step in scene analysis algorithms. The recent success of deep learning envisions a new generation of despeckling techniques that could outperform classical model-based methods. However, current deep learning approaches to despeckling require supervision for training, whereas clean SAR images are impossible to obtain. In the literature, this issue is tackled by resorting to either synthetically speckled optical images, which exhibit different propert…

2020-07-04cs.CVeess.IV

Towards Deep Unsupervised SAR Despeckling with Blind-Spot Convolutional Neural Networks

PoliTO（都灵理工大学）

SAR despeckling is a problem of paramount importance in remote sensing, since it represents the first step of many scene analysis algorithms. Recently, deep learning techniques have outperformed classical model-based despeckling algorithms. However, such methods require clean ground truth images for training, thus resorting to synthetically speckled optical images since clean SAR images cannot be acquired. In this paper, inspired by recent works on blind-spot denoising networks, we propose a self-supervised Bayesian despeckling method. The proposed method is trained employing only noisy images…

2020-01-15cs.CVcs.LGeess.IV

AP-BSN: Self-Supervised Denoising for Real-World Images via Asymmetric PD and Blind-Spot Network

Seoul National University（首尔国立大学）

Blind-spot network (BSN) and its variants have made significant advances in self-supervised denoising. Nevertheless, they are still bound to synthetic noisy inputs due to less practical assumptions like pixel-wise independent noise. Hence, it is challenging to deal with spatially correlated real-world noise using self-supervised BSN. Recently, pixel-shuffle downsampling (PD) has been proposed to remove the spatial correlation of real-world noise. However, it is not trivial to integrate PD and BSN directly, which prevents the fully self-supervised denoising model on real-world images. We propos…

2022-03-24cs.CVeess.IV

Masked Face Inpainting Through Residual Attention UNet

Bahcesehir University（巴耶塞特大学）；American University of Malta（马耳他美国大学）

Realistic image restoration with high texture areas such as removing face masks is challenging. The state-of-the-art deep learning-based methods fail to guarantee high-fidelity, cause training instability due to vanishing gradient problems (e.g., weights are updated slightly in initial layers) and spatial information loss. They also depend on intermediary stage such as segmentation meaning require external mask. This paper proposes a blind mask face inpainting method using residual attention UNet to remove the face mask and restore the face with fine details while minimizing the gap with the g…

2022-09-19cs.CVeess.IV

Improving Blind Spot Denoising for Microscopy

Max-Planck Institute of Molecular Cell Biology and Genetics（马克斯·普朗克分子细胞生物学与遗传学研究所）；Center for Systems Biology Dresden (CSBD)（德累斯顿系统生物学中心）；Fondazione Human Technopole（人类技术城基金会）；Max Planck Institute for the Physics of Complex Systems（马克斯·普朗克复杂系统物理研究所）

Many microscopy applications are limited by the total amount of usable light and are consequently challenged by the resulting levels of noise in the acquired images. This problem is often addressed via (supervised) deep learning based denoising. Recently, by making assumptions about the noise statistics, self-supervised methods have emerged. Such methods are trained directly on the images that are to be denoised and do not require additional paired training data. While achieving remarkable results, self-supervised methods can produce high-frequency artifacts and achieve inferior results compar…

2020-08-19cs.CVeess.IV

Domino Denoise: An Accurate Blind Zero-Shot Denoiser using Domino Tilings

Lunenfeld-Tanenbaum Research Institute（卢嫩菲尔德-塔南鲍姆研究所）；University of Toronto（多伦多大学）

Because noise can interfere with downstream analysis, image denoising has come to occupy an important place in the image processing toolbox. The most accurate state-of-the-art denoisers typically train on a representative dataset. But gathering a training set is not always feasible, so interest has grown in blind zero-shot denoisers that train only on the image they are denoising. The most accurate blind-zero shot methods are blind-spot networks, which mask pixels and attempt to infer them from their surroundings. Other methods exist where all neurons participate in forward inference, however …

2022-12-05cs.CVeess.IV

消除盲区：基于扩散模型的子宫MRI合成

Diffusing the Blind Spot: Uterine MRI Synthesis with Diffusion Models

Imperial College London, London, UK（伦敦帝国理工学院）

针对现有扩散模型难以生成解剖学精确的女性盆腔图像的问题，提出整合2D/3D无条件及条件DDPMs与LDMs的扩散框架，生成高保真子宫MRI合成图像，提升诊断模型准确性并发布数据集推动妇科AI研究。

2025-08-25cs.AIcs.CVeess.IV

MGAN-CRCM：一种用于图像修复的新型多重生成对抗网络与粗-精炼认知方法

MGAN-CRCM: A Novel Multiple Generative Adversarial Network and Coarse-Refinement Based Cognizant Method for Image Inpainting

Bangladesh University of Professionals（孟加拉国专业大学）；Jahangirnagar University（贾汉吉尔纳加尔大学）；University of New South Wales (UNSW)（新南威尔士大学（UNSW））；Charles Sturt University（查尔斯特大学）

本文提出MGAN-CRCM架构，结合多重GAN与ResNet进行图像修复，通过转置卷积GAN、FR-CNN和Co-Mod GAN分别实现盲修复、目标移除和细化，在多个基准数据集上取得优于现有方法的准确率。

2024-12-25cs.CVcs.LGeess.IV