In many real-world applications, image data often come with noises, corruptions or large errors. One approach to deal with noise image data is to use data recovery techniques which aim to recover the true uncorrupted signals from the observed noise images. In this paper, we first introduce a novel corruption recovery transformation (CRT) model which aims to recover multiple (or a collection of) corrupted images using a single affine transformation. Then, we show that the introduced CRT can be efficiently constructed through learning from training data. Once CRT is learned, we can recover the true signals from the new incoming/test corrupted images explicitly. As an application, we apply our CRT to image recognition task. Experimental results on six image datasets demonstrate that the proposed CRT model is effective in recovering noise image data and thus leads to better recognition results.
Region Feature Descriptor Adapted to High Affine Transformations
School of Artificial Intelligence and Computer Science, Jiangnan University（江南大学人工智能与计算机科学学院）；Engineering Research Center of Intelligent Technology for Healthcare, Ministry of Education（教育部智慧医疗技术工程研究中心）
Instituto de Matemáticas, UNAM（墨西哥国立自治大学数学研究所）；Facultad de Ciencias Físico-Matemáticas, UMSNH（米却肯州圣尼古拉斯大学物理数学科学学院）
Given a collection $\{Z_1,Z_2,\ldots,Z_m\}$ of $n$-sided polygons in the plane and a query polygon $W$ we give algorithms to find all $Z_\ell$ such that $W=f(Z_\ell)$ with $f$ an unknown similarity transformation in time independent of the size of the collection. If $f$ is a known affine transformation, we show how to find all $Z_\ell$ such that $W=f(Z_\ell)$ in $O(n+\log(m))$ time. For a pair $W,W^\prime$ of polygons we can find all the pairs $Z_\ell,Z_{\ell^\prime}$ such that $W=f(Z_\ell)$ and $W^\prime=f(Z_{\ell^\prime})$ for an unknown affine transformation $f$ in $O(m+n)$ time. For the ca…
This paper presents a deep architecture for dense semantic correspondence, called pyramidal affine regression networks (PARN), that estimates locally-varying affine transformation fields across images. To deal with intra-class appearance and shape variations that commonly exist among different instances within the same object category, we leverage a pyramidal model where affine transformation fields are progressively estimated in a coarse-to-fine manner so that the smoothness constraint is naturally imposed within deep networks. PARN estimates residual affine transformations at each level and …
University of Munich（慕尼黑大学）；Siemens AG, Corporate Technology（西门子股份公司企业技术部）
Convolutional neural networks (CNNs) achieve translational invariance by using pooling operations. However, the operations do not preserve the spatial relationships in the learned representations. Hence, CNNs cannot extrapolate to various geometric transformations of inputs. Recently, Capsule Networks (CapsNets) have been proposed to tackle this problem. In CapsNets, each entity is represented by a vector and routed to high-level entity representations by a dynamic routing algorithm. CapsNets have been shown to be more robust than CNNs to affine transformations of inputs. However, there is sti…
Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）；University of Chinese Academy of Sciences（中国科学院大学）；Microsoft AI & Research Group（微软人工智能与研究事业部）；Microsoft Search Technology Center Asia（微软亚洲搜索技术中心）
We proposed a kind of naturally combined shape-color affine moment invariants (SCAMI), which consider both shape and color affine transformations simultaneously in one single system. In the real scene, color and shape deformations always exist in images simultaneously. Simple shape invariants or color invariants can not be qualified for this situation. The conventional method is just to make a simple linear combination of the two factors. Meanwhile, the manual selection of weights is a complex issue. Our construction method is based on the multiple integration framework. The integral kernel is…
Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）
We propose the general construction formula of shape-color primitives by using partial differentials of each color channel in this paper. By using all kinds of shape-color primitives, shape-color differential moment invariants can be constructed very easily, which are invariant to the shape affine and color affine transforms. 50 instances of SCDMIs are obtained finally. In experiments, several commonly used color descriptors and SCDMIs are used in image classification and retrieval of color images, respectively. By comparing the experimental results, we find that SCDMIs get better results.
Text-to-image synthesis aims to generate natural images conditioned on text descriptions. The main difficulty of this task lies in effectively fusing text information into the image synthesis process. Existing methods usually adaptively fuse suitable text information into the synthesis process with multiple isolated fusion blocks (e.g., Conditional Batch Normalization and Instance Normalization). However, isolated fusion blocks not only conflict with each other but also increase the difficulty of training (see first page of the supplementary). To address these issues, we propose a Recurrent Af…
Yonsei University（延世大学）；Chungnam National University（忠南国立大学）；Microsoft Research（微软研究院）
Techniques for dense semantic correspondence have provided limited ability to deal with the geometric variations that commonly exist between semantically similar images. While variations due to scale and rotation have been examined, there lack practical solutions for more complex deformations such as affine transformations because of the tremendous size of the associated solution space. To address this problem, we present a discrete-continuous transformation matching (DCTM) framework where dense affine transformation fields are inferred through a discrete label optimization in which the labels…
This study attempts to provide explanations, descriptions and evaluations of some most popular and current combinations of description and descriptor frameworks, namely SIFT, SURF, MSER, and BRISK for keypoint extractors and SIFT, SURF, BRISK, and FREAK for descriptors. Evaluations are made based on the number of matches of keypoints and repeatability in various image variations. It is used as the main parameter to assess how well combinations of algorithms are in matching objects with different variations. There are many papers that describe the comparison of detection and description feature…
Affine transformation is one of the most common transformations in nature, which is an important issue in the field of computer vision and shape analysis. And affine transformations often occur in both shape and color space simultaneously, which can be termed as Dual-Affine Transformation (DAT). In general, we should derive invariants of different data formats separately, such as 2D color images, 3D color objects, or even higher-dimensional data. To the best of our knowledge, there is no general framework to derive invariants for all of these data formats. In this paper, we propose a general f…
Relationships between the degrees of freedom in the affine Gaussian derivative model for visual receptive fields and 2-D affine image transformations, with application to covariance properties of simple cells in the primary visual cortex
Computational Brain Science Lab（计算脑科学实验室）；Division of Computational Science and Technology（计算科学与技术系）；KTH Royal Institute of Technology（皇家理工学院）
Sapienza University of Rome（罗马大学）；Fondazione Bruno Kessler（布鲁诺·凯塞勒基金会）；University of Trento（特伦托大学）；Politecnico di Torino（都灵理工大学）；Italian Institute of Technology（意大利技术研究院）；Mapillary Research（Mapillary研究部门）
In this work, we present a new, algorithm for multi-domain learning. Given a pretrained architecture and a set of visual domains received sequentially, the goal of multi-domain learning is to produce a single model performing a task in all the domains together. Recent works showed how we can address this problem by masking the internal weights of a given original conv-net through learned binary variables. In this work, we provide a general formulation of binary mask based models for multi-domain learning by affine transformations of the original network parameters. Our formulation obtains sign…
University of Queensland（昆士兰大学）；Central South University（中南大学）
Deep neural networks have demonstrated great potential in solving dipole inversion for Quantitative Susceptibility Mapping (QSM). However, the performances of most existing deep learning methods drastically degrade with mismatched sequence parameters such as acquisition orientation and spatial resolution. We propose an end-to-end AFfine Transformation Edited and Refined (AFTER) deep neural network for QSM, which is robust against arbitrary acquisition orientation and spatial resolution up to 0.6 mm isotropic at the finest. The AFTER-QSM neural network starts with a forward affine transformatio…
We introduce an end-to-end deep-learning framework for 3D medical image registration. In contrast to existing approaches, our framework combines two registration methods: an affine registration and a vector momentum-parameterized stationary velocity field (vSVF) model. Specifically, it consists of three stages. In the first stage, a multi-step affine network predicts affine transform parameters. In the second stage, we use a Unet-like network to generate a momentum, from which a velocity field can be computed via smoothing. Finally, in the third stage, we employ a self-iterable map-based vSVF …
In recent years, unsupervised domain adaptation (UDA) for semantic segmentation has brought many researchers'attention. Many of them take an approach to design a complex system so as to better align the gap between source and target domain. Instead, we focus on the very basic structure of the deep neural network, Batch Normalization, and propose to replace the Sharing Affine Transformation with our proposed Separate Affine Transformation (SEAT). The proposed SEAT is simple, easily implemented and easy to integrate into existing adversarial learning based UDA methods. Also, to further improve t…
Given an affine transformation $T$, we define its Fisher distortion $Dist_F(T)$. We show that the Fisher distortion has Riemannian metric structure and provide an algorithm for finding mean distorting transformation -- namely -- for a given set $\{T_{i}\}_{i=1}^N$ of affine transformations, find an affine transformation $T$ that minimize the overall distortion $\sum_{i=1}^NDist_F^{2}(T^{-1}T_{i}).$ The mean distorting transformation can be useful in some fields -- in particular, we apply it for rendering affine panoramas.
Tsinghua University（清华大学）；Beijing National Research Center for Information Science and Technology（北京信息科学与技术国家研究中心）
Conventional point cloud semantic segmentation methods usually employ an encoder-decoder architecture, where mid-level features are locally aggregated to extract geometric information. However, the over-reliance on these class-agnostic local geometric representations may raise confusion between local parts from different categories that are similar in appearance or spatially adjacent. To address this issue, we argue that mid-level features can be further enhanced with semantic information, and propose semantic-affine transformation that transforms features of mid-level points belonging to diff…
University of Bucharest（布加勒斯特大学）；SecurifAI；Romanian Young Academy（罗马尼亚青年科学院）
Recent studies revealed that convolutional neural networks do not generalize well to small image transformations, e.g. rotations by a few degrees or translations of a few pixels. To improve the robustness to such transformations, we propose to introduce data augmentation at intermediate layers of the neural architecture, in addition to the common data augmentation applied on the input images. By introducing small perturbations to activation maps (features) at various levels, we develop the capacity of the neural network to cope with such transformations. We conduct experiments on three image c…
Adversarial attacks are small, carefully crafted perturbations, imperceptible to the naked eye; that when added to an image cause deep learning models to misclassify the image with potentially detrimental outcomes. With the rise of artificial intelligence models in consumer safety and security intensive industries such as self-driving cars, camera surveillance and face recognition, there is a growing need for guarding against adversarial attacks. In this paper, we present an approach that uses metamorphic testing principles to automatically detect such adversarial attacks. The approach can det…
Reliable feature point matching is a vital yet challenging process in feature-based image registration. In this paper,a robust feature point matching algorithm called Recovery and Filtering Vertex Trichotomy Matching (RFVTM) is proposed to remove outliers and retain sufficient inliers for remote sensing images. A novel affine invariant descriptor called vertex trichotomy descriptor is proposed on the basis of that geometrical relations between any of vertices and lines are preserved after affine transformations, which is constructed by mapping each vertex into trichotomy sets. The outlier remo…
Shanghai Jiao Tong University（上海交通大学）；OPPO Research Institute（OPPO研究院）；Tsinghua University（清华大学）
As designers of artificial intelligence try to outwit hackers, both sides continue to hone in on AI's inherent vulnerabilities. Designed and trained from certain statistical distributions of data, AI's deep neural networks (DNNs) remain vulnerable to deceptive inputs that violate a DNN's statistical, predictive assumptions. Before being fed into a neural network, however, most existing adversarial examples cannot maintain malicious functionality when applied to an affine transformation. For practical purposes, maintaining that malicious functionality serves as an important measure of the robus…
TUV SUD Singapore（新加坡TUV南德意志集团）；Nanyang Technological University（南洋理工大学）
Autonomous vehicles (AV) have progressed rapidly with the advancements in computer vision algorithms. The deep convolutional neural network as the main contributor to this advancement has boosted the classification accuracy dramatically. However, the discovery of adversarial examples reveals the generalization gap between dataset and the real world. Furthermore, affine transformations may also confuse computer vision based object detectors. The degradation of the perception system is undesirable for safety critical systems such as autonomous vehicles. In this paper, a deep learning system is p…
