Point-DAE: Denoising Autoencoders for Self-supervised Point Cloud Learning
Department of Computing, The Hong Kong Polytechnic University（计算系，香港理工大学）；School of Electronic and Information Engineering, South China University of Technology（电子与信息工程学院，华南理工大学）
Yabin Zhang, Jiehong Lin, Ruihuang Li, Kui Jia, Lei Zhang
Masked autoencoder has demonstrated its effectiveness in self-supervised point cloud learning. Considering that masking is a kind of corruption, in this work we explore a more general denoising autoencoder for point cloud learning (Point-DAE) by investigating more types of corruptions beyond masking. Specifically, we degrade the point cloud with certain corruptions as input, and learn an encoder-decoder model to reconstruct the original point cloud from its corrupted version. Three corruption families (\ie, density/masking, noise, and affine transformation) and a total of fourteen corruption types are investigated with traditional non-Transformer encoders. Besides the popular masking corruption, we identify another effective corruption family, \ie, affine transformation. The affine transformation disturbs all points globally, which is complementary to the masking corruption where some local regions are dropped. We also validate the effectiveness of affine transformation corruption with the Transformer backbones, where we decompose the reconstruction of the complete point cloud into the reconstructions of detailed local patches and rough global shape, alleviating the position leakage problem in the reconstruction. Extensive experiments on tasks of object classification, few-shot learning, robustness testing, part segmentation, and 3D object detection validate the effectiveness of the proposed method. The codes are available at \url{https://github.com/YBZh/Point-DAE}.
DAE-Fuse: An Adaptive Discriminative Autoencoder for Multi-Modality Image Fusion
Department of Computer Science（计算机科学系）；Guangdong Provincial Key Lab of Interdisciplinary Research and Application for Data Science（广东省交叉学科研究与数据科学应用重点实验室）；Beijing Normal - Hong Kong Baptist University（北京师范大学-香港 Baptist大学）
Institute for Signals, Systems and Computational Intelligence, sinc(i) CONICET-UNL（信号、系统与计算智能研究所，sinc(i) 阿根廷国家科学与技术研究理事会-国立利托拉尔大学）；Imperial College London（伦敦帝国理工学院）
We introduce Post-DAE, a post-processing method based on denoising autoencoders (DAE) to improve the anatomical plausibility of arbitrary biomedical image segmentation algorithms. Some of the most popular segmentation methods (e.g. based on convolutional neural networks or random forest classifiers) incorporate additional post-processing steps to ensure that the resulting masks fulfill expected connectivity constraints. These methods operate under the hypothesis that contiguous pixels with similar aspect should belong to the same class. Even if valid in general, this assumption does not consid…
Imperial College London（帝国理工学院）；Data Science Institute, Imperial College London（帝国理工学院数据科学研究所）
Denoising autoencoders (DAEs) are powerful deep learning models used for feature extraction, data generation and network pre-training. DAEs consist of an encoder and decoder which may be trained simultaneously to minimise a loss (function) between an input and the reconstruction of a corrupted version of the input. There are two common loss functions used for training autoencoders, these include the mean-squared error (MSE) and the binary cross-entropy (BCE). When training autoencoders on image data a natural choice of loss function is BCE, since pixel values may be normalised to take values i…
Industrial Systems Institute, Athena Research Center（雅典研究中心工业系统研究所）；Department of Electrical & Computer Engineering University of Patras（帕特拉斯大学电气与计算机工程系）
Recent advances in 3D scanning technology have enabled the deployment of 3D models in various industrial applications like digital twins, remote inspection and reverse engineering. Despite their evolving performance, 3D scanners, still introduce noise and artifacts in the acquired dense models. In this work, we propose a fast and robust denoising method for dense 3D scanned industrial models. The proposed approach employs conditional variational autoencoders to effectively filter face normals. Training and inference are performed in a sliding patch setup reducing the size of the required train…
Detecting Unforeseen Data Properties with Diffusion Autoencoder Embeddings using Spine MRI data
Klinikum rechts der Isar（Isar临床中心）；TUM School of Medicine and Health（慕尼黑工业大学医学与健康学院）；Institut fuer KI und Informatik in der Medizin（医学人工智能与信息学研究所）；German National Cohort（德国国家队列研究中心）
Fraunhofer Institute for Computer Graphics Research (IGD)（弗劳恩霍夫计算机图形研究所）；University of Rostock（罗斯托克大学）
Visual inspection of underwater structures by vehicles, e.g. remotely operated vehicles (ROVs), plays an important role in scientific, military, and commercial sectors. However, the automatic extraction of information using software tools is hindered by the characteristics of water which degrade the quality of captured videos. As a contribution for restoring the color of underwater images, Underwater Denoising Autoencoder (UDAE) model is developed using a denoising autoencoder with U-Net architecture. The proposed network takes into consideration the accuracy and the computation cost to enable…
College of Information and Electrical Engineering, China Agricultural University（中国农业大学信息与电气工程学院）
In Synthetic Aperture Radar (SAR) imaging, despeckling is very important for image analysis,whereas speckle is known as a kind of multiplicative noise caused by the coherent imaging system. During the past three decades, various algorithms have been proposed to denoise the SAR image. Generally, the BM3D is considered as the state of art technique to despeckle the speckle noise with excellent performance. More recently, deep learning make a success in image denoising and achieved a improvement over conventional method where large train dataset is required. Unlike most of the images SAR image de…
Suitable lateral connections between encoder and decoder are shown to allow higher layers of a denoising autoencoder (dAE) to focus on invariant representations. In regular autoencoders, detailed information needs to be carried through the highest layers but lateral connections from encoder to decoder relieve this pressure. It is shown that abstract invariant features can be translated to detailed reconstructions when invariant features are allowed to modulate the strength of the lateral connection. Three dAE structures with modulated and additive lateral connections, and without lateral conne…
Image denoising is an important pre-processing step in medical image analysis. Different algorithms have been proposed in past three decades with varying denoising performances. More recently, having outperformed all conventional methods, deep learning based models have shown a great promise. These methods are however limited for requirement of large training sample size and high computational costs. In this paper we show that using small sample size, denoising autoencoders constructed using convolutional layers can be used for efficient denoising of medical images. Heterogeneous images can be…
Noise in speckle-prone optical coherence tomography tends to obfuscate important details necessary for medical diagnosis. In this paper, a denoising approach that preserves disease characteristics on retinal optical coherence tomography images in ophthalmology is presented. By combining a deep convolutional autoencoder with a priorly trained ResNet image classifier as regularizer, the perceptibility of delicate details is encouraged and only information-less background noise is filtered out. With our approach, higher peak signal-to-noise ratios with $ \mathrm{PSNR} = 31.2\,\mathrm{dB} $ and hi…
Image denoising based on a probabilistic model of local image patches has been employed by various researchers, and recently a deep (denoising) autoencoder has been proposed by Burger et al. [2012] and Xie et al. [2012] as a good model for this. In this paper, we propose that another popular family of models in the field of deep learning, called Boltzmann machines, can perform image denoising as well as, or in certain cases of high level of noise, better than denoising autoencoders. We empirically evaluate the two models on three different sets of images with different types and levels of nois…
Nowadays, Deep learning techniques show dramatic performance on computer vision area, and they even outperform human. But it is also vulnerable to some small perturbation called an adversarial attack. This is a problem combined with the safety of artificial intelligence, which has recently been studied a lot. These attacks have shown that they can fool models of image classification, semantic segmentation, and object detection. We point out this attack can be protected by denoise autoencoder, which is used for denoising the perturbation and restoring the original images. We experiment with var…
Unsupervised learning is of growing interest because it unlocks the potential held in vast amounts of unlabelled data to learn useful representations for inference. Autoencoders, a form of generative model, may be trained by learning to reconstruct unlabelled input data from a latent representation space. More robust representations may be produced by an autoencoder if it learns to recover clean input samples from corrupted ones. Representations may be further improved by introducing regularisation during training to shape the distribution of the encoded data in latent space. We suggest denois…
We propose a novel deep learning model for classifying medical images in the setting where there is a large amount of unlabelled medical data available, but labelled data is in limited supply. We consider the specific case of classifying skin lesions as either malignant or benign. In this setting, the proposed approach -- the semi-supervised, denoising adversarial autoencoder -- is able to utilise vast amounts of unlabelled data to learn a representation for skin lesions, and small amounts of labelled data to assign class labels based on the learned representation. We analyse the contributions…
The Biorobotics Institute（生物机器人研究所）；Scuola Superiore Sant’Anna（圣安娜高等学校）
Despite the recent developments that allowed neural networks to achieve impressive performance on a variety of applications, these models are intrinsically affected by the problem of overgeneralization, due to their partitioning of the full input space into the fixed set of target classes used during training. Thus it is possible for novel inputs belonging to categories unknown during training or even completely unrecognizable to humans to fool the system into classifying them as one of the known classes, even with a high degree of confidence. Solving this problem may help improve the security…
Physics-Constrained Denoising Autoencoders for Data-Scarce Wildfire UAV Sensing
FEWALab and Geological Sciences and Geological Engineering, Queen’s University（FEWALab及地质科学与地质工程、女王大学）；School of Environmental Studies（环境研究学院；地质科学与地质工程、女王大学）；Geological Sciences and Geological Engineering, Queen’s University
National University of Defense Technology（国防科技大学）；Simon Fraser University（西蒙弗雷泽大学）；Princeton University（普林斯顿大学）
Indoor scenes exhibit rich hierarchical structure in 3D object layouts. Many tasks in 3D scene understanding can benefit from reasoning jointly about the hierarchical context of a scene, and the identities of objects. We present a variational denoising recursive autoencoder (VDRAE) that generates and iteratively refines a hierarchical representation of 3D object layouts, interleaving bottom-up encoding for context aggregation and top-down decoding for propagation. We train our VDRAE on large-scale 3D scene datasets to predict both instance-level segmentations and a 3D object detections from an…
Research institute for signals, systems and computational intelligence, sinc(i)（信号、系统与计算智能研究所（sinc(i)））；FICH-UNL（拉普拉塔国立大学工程学院）；CONICET（阿根廷国家科学与技术研究理事会）
Deep convolutional neural networks (CNN) proved to be highly accurate to perform anatomical segmentation of medical images. However, some of the most popular CNN architectures for image segmentation still rely on post-processing strategies (e.g. Conditional Random Fields) to incorporate connectivity constraints into the resulting masks. These post-processing steps are based on the assumption that objects are usually continuous and therefore nearby pixels should be assigned the same object label. Even if it is a valid assumption in general, these methods do not offer a straightforward way to in…
Indian Institute of Technology Madras（印度理工学院马德拉斯分校）；Sree Chitra Tirunal Institute for Medical Sciences and Technology（Sree Chitra Tirunal医学科学与技术研究所）
The work presented explores the use of denoising autoencoders (DAE) for brain lesion detection, segmentation and false positive reduction. Stacked denoising autoencoders (SDAE) were pre-trained using a large number of unlabeled patient volumes and fine tuned with patches drawn from a limited number of patients (n=20, 40, 65). The results show negligible loss in performance even when SDAE was fine tuned using 20 patients. Low grade glioma (LGG) segmentation was achieved using a transfer learning approach wherein a network pre-trained with High Grade Glioma (HGG) data was fine tuned using LGG im…
Herein, we present a system for hyperspectral image segmentation that utilizes multiple class--based denoising autoencoders which are efficiently trained. Moreover, we present a novel hyperspectral data augmentation method for labelled HSI data using linear mixtures of pixels from each class, which helps the system with edge pixels which are almost always mixed pixels. Finally, we utilize a deep neural network and morphological hole-filling to provide robust image classification. Results run on the Salinas dataset verify the high performance of the proposed algorithm.
Low-dose computed tomography (LDCT) reduces the X-ray radiation but compromises image quality with more noises and artifacts. A plethora of transformer models have been developed recently to improve LDCT image quality. However, the success of a transformer model relies on a large amount of paired noisy and clean data, which is often unavailable in clinical applications. In computer vision and natural language processing fields, masked autoencoders (MAE) have been proposed as an effective label-free self-pretraining method for transformers, due to its excellent feature representation ability. H…
Technical University of Munich（慕尼黑工业大学）；Klinikum Rechts der Isar (Technical University of Munich)（慕尼黑工业大学伊萨尔河右岸医院）；University of Munich（慕尼黑大学）；Klinikum der Universität München（慕尼黑大学医院）；Johns Hopkins University（约翰斯·霍普金斯大学）
Vertebral fractures are a consequence of osteoporosis, with significant health implications for affected patients. Unfortunately, grading their severity using CT exams is hard and subjective, motivating automated grading methods. However, current approaches are hindered by imbalance and scarcity of data and a lack of interpretability. To address these challenges, this paper proposes a novel approach that leverages unlabelled data to train a generative Diffusion Autoencoder (DAE) model as an unsupervised feature extractor. We model fracture grading as a continuous regression, which is more refl…
Indraprastha Institute of Information Technology（英迪拉·普拉斯特哈信息技术研究所）
In this work we address the problem of real-time dynamic MRI reconstruction. There are a handful of studies on this topic; these techniques are either based on compressed sensing or employ Kalman Filtering. These techniques cannot achieve the reconstruction speed necessary for real-time reconstruction. In this work, we propose a new approach to MRI reconstruction. We learn a non-linear mapping from the unstructured aliased images to the corresponding clean images using a stacked denoising autoencoder (SDAE). The training for SDAE is slow, but the reconstruction is very fast - only requiring a …
Worcester Polytechnic Institute（伍斯特理工学院）；Colorado State University（科罗拉多州立大学）
In this paper, we demonstrate how deep autoencoders can be generalized to the case of inpainting and denoising, even when no clean training data is available. In particular, we show how neural networks can be trained to perform all of these tasks simultaneously. While, deep autoencoders implemented by way of neural networks have demonstrated potential for denoising and anomaly detection, standard autoencoders have the drawback that they require access to clean data for training. However, recent work in Robust Deep Autoencoders (RDAEs) shows how autoencoders can be trained to eliminate outliers…
Hyperspectral image analysis often requires selecting the most informative bands instead of processing the whole data without losing the key information. Existing band reduction (BR) methods have the capability to reveal the nonlinear properties exhibited in the data but at the expense of loosing its original representation. To cope with the said issue, an unsupervised non-linear segmented and non-segmented stacked denoising autoencoder (UDAE) based BR method is proposed. Our aim is to find an optimal mapping and construct a lower-dimensional space that has a similar structure to the original …
This paper presents a fractional one-dimensional convolutional neural network (CNN) autoencoder for denoising the Electroencephalogram (EEG) signals which often get contaminated with noise during the recording process, mostly due to muscle artifacts (MA), introduced by the movement of muscles. The existing EEG denoising methods make use of decomposition, thresholding and filtering techniques. In the proposed approach, EEG signals are first transformed to orthogonal domain using Tchebichef moments before feeding to the proposed architecture. A new hyper-parameter ($α$) is introduced which refer…
Collaborative Auto-encoding for Blind Image Quality Assessment
Shenzhen University（深圳大学）；Peng Cheng Laboratory（鹏城实验室）；Shenzhen Institute for Artificial Intelligence and Robotics for Society（深圳人工智能与机器人研究院）；University of Nottingham（诺丁汉大学）
Energy-based models parameterize the unnormalized log-probability of data samples, but there is a lack of guidance on how to construct the "energy". In this paper, we propose a Denoising-EBM which decomposes the image energy into "semantic energy" and "texture energy". We define the "semantic energy" in the latent space of DAE to model the high-level representations, and define the pixel-level reconstruction error for denoising as "texture energy". Inspired by score-based model, our model utilizes multi-scale noisy samples for maximum-likelihood training and it outputs a vector instead of a sc…
The success of deep architectures is at least in part attributed to the layer-by-layer unsupervised pre-training that initializes the network. Various papers have reported extensive empirical analysis focusing on the design and implementation of good pre-training procedures. However, an understanding pertaining to the consistency of parameter estimates, the convergence of learning procedures and the sample size estimates is still unavailable in the literature. In this work, we study pre-training in classical and distributed denoising autoencoders with these goals in mind. We show that the grad…
Hefei University（合肥学院）；School of Artificial Intelligence and Big Data, Hefei University（合肥学院人工智能与大数据学院）；Institute of Applied Optimization, School of Artificial Intelligence and Big Data, Hefei University（合肥学院人工智能与大数据学院应用优化研究所）；School of Computer Science and Technology, University of Science and Technology of China（中国科学技术大学计算机科学与技术学院）；University of Science and Technology of China（中国科学技术大学）
Recently, with the availability of cost-effective depth cameras coupled with real-time skeleton estimation, the interest in skeleton-based human action recognition is renewed. Most of the existing skeletal representation approaches use either the joint location or the dynamics model. Differing from the previous studies, we propose a new method called Denoising Autoencoder with Temporal and Categorical Constraints (DAE_CTC)} to study the skeletal representation in a view of skeleton reconstruction. Based on the concept of learning under privileged information, we integrate action categories and…
The neighborhood effect is a key driving factor for the land-use change (LUC) process. This study applies convolutional neural networks (CNN) to capture neighborhood characteristics from satellite images and to enhance the performance of LUC modeling. We develop a hybrid CNN model (conv-net) to predict the LU transition probability by combining satellite images and geographical features. A spatial weight layer is designed to incorporate the distance-decay characteristics of neighborhood effect into conv-net. As an alternative model, we also develop a hybrid convolutional denoising autoencoder …
Denoising Vision Transformer Autoencoder with Spectral Self-Regularization
Nanjing University（南京大学）；Kling Team, Kuaishou Technology（快手科技 Kling 团队）；Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））；Shanghai Jiao Tong University（上海交通大学）；University of Science and Technology of China（中国科学技术大学）
DRACO: A Denoising-Reconstruction Autoencoder for Cryo-EM
School of Information Science and Technology, ShanghaiTech University（上海科技大学信息科学与技术学院）；Cellverse Co, Ltd.（Cellverse公司）；iHuman Institute, ShanghaiTech University（上海科技大学iHuman研究所）
With the inexorable digitalisation of the modern world, every subset in the field of technology goes through major advancements constantly. One such subset is digital images which are ever so popular. Images can not always be as visually pleasing or clear as you would want them to be and are often distorted or obscured with noise. A number of techniques to enhance images have come up as the years passed, all with their own respective pros and cons. In this paper, we look at one such particular technique which accomplishes this task with the help of a neural network model commonly known as an a…
VRAE: Vertical Residual Autoencoder for License Plate Denoising and Deblurring
Ha Noi University of Science and Technology（河内科学技术大学）；Center of Environmental Intelligence（环境智能中心）；VinUniversity（文大学）；University of Southern California（南加州大学）
Counterfactual Explanations for Medical Image Classification and Regression using Diffusion Autoencoder
Technical University of Munich（慕尼黑工业大学）；Klinikum rechts der Isar（右侧伊萨尔医院）；Institute for Artificial Intelligence and Computer Science in Medicine, Technical University of Munich（慕尼黑工业大学人工智能与医学计算机科学研究所）；Computer Aided Medical Procedures, Technical University of Munich（慕尼黑工业大学计算机辅助医疗程序研究所）；AI for Image-Guided Diagnosis and Therapy, Technical University of Munich（慕尼黑工业大学图像引导诊断与治疗人工智能研究所）
University Paris-Est（巴黎东大学）；Ecole des Ponts ParisTech（巴黎高科桥路学院）
Given an initial recognition model already trained on a set of base classes, the goal of this work is to develop a meta-model for few-shot learning. The meta-model, given as input some novel classes with few training examples per class, must properly adapt the existing recognition model into a new model that can correctly classify in a unified way both the novel and the base classes. To accomplish this goal it must learn to output the appropriate classification weight vectors for those two types of classes. To build our meta-model we make use of two main innovations: we propose the use of a De…
An alternative pathway for the human brain to communicate with the outside world is by means of a brain computer interface (BCI). A BCI can decode electroencephalogram (EEG) signals of brain activities, and then send a command or an intent to an external interactive device, such as a wheelchair. The effectiveness of the BCI depends on the performance in decoding the EEG. Usually, the EEG is contaminated by different kinds of artefacts (e.g., electromyogram (EMG), background activity), which leads to a low decoding performance. A number of filtering methods can be utilized to remove or weaken t…
Peking University（北京大学）；Microsoft Research Asia（微软亚洲研究院）
In this paper, we propose a new self-supervised method, which is called Denoising Masked AutoEncoders (DMAE), for learning certified robust classifiers of images. In DMAE, we corrupt each image by adding Gaussian noises to each pixel value and randomly masking several patches. A Transformer-based encoder-decoder model is then trained to reconstruct the original image from the corrupted one. In this learning paradigm, the encoder will learn to capture relevant semantics for the downstream tasks, which is also robust to Gaussian additive noises. We show that the pre-trained encoder can naturally…
University of Oxford（牛津大学）；Tencent AI Lab（腾讯人工智能实验室）；University of Technology Sydney（悉尼科技大学）；South China University of Technology（华南理工大学）；University of Oregon（俄勒冈大学）；City University of Hong Kong（香港城市大学）；University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）
While deep neural networks have been shown to perform remarkably well in many machine learning tasks, labeling a large amount of ground truth data for supervised training is usually very costly to scale. Therefore, learning robust representations with unlabeled data is critical in relieving human effort and vital for many downstream tasks. Recent advances in unsupervised and self-supervised learning approaches for visual data have benefited greatly from domain knowledge. Here we are interested in a more generic unsupervised learning framework that can be easily generalized to other domains. In…
Center for Systems Biology Dresden（德累斯顿系统生物学中心）；Max-Planck Institute (CBG)（马克斯·普朗克分子细胞生物学与遗传学研究所）；School of Computer Science（计算机科学学院）；University of Birmingham（伯明翰大学）；Fondazione Human Technopole（人类技术基金会）
Deep Learning based methods have emerged as the indisputable leaders for virtually all image restoration tasks. Especially in the domain of microscopy images, various content-aware image restoration (CARE) approaches are now used to improve the interpretability of acquired data. Naturally, there are limitations to what can be restored in corrupted images, and like for all inverse problems, many potential solutions exist, and one of them must be chosen. Here, we propose DivNoising, a denoising approach based on fully convolutional variational autoencoders (VAEs), overcoming the problem of havin…
