Alignment between non-rigid stretchable structures is one of the most challenging tasks in computer vision, as the invariant properties are hard to define, and there is no labeled data for real datasets. We present unsupervised neural network architecture based upon the spectral domain of scale-invariant geometry. We build on top of the functional maps architecture, but show that learning local features, as done until now, is not enough once the isometry assumption breaks. We demonstrate the use of multiple scale-invariant geometries for solving this problem. Our method is agnostic to local-scale deformations and shows superior performance for matching shapes from different domains when compared to existing spectral state-of-the-art solutions.
Zhejiang University（浙江大学）；University of Twente（特文特大学）
Multimodal information (e.g., visible and thermal) can generate robust pedestrian detections to facilitate around-the-clock computer vision applications, such as autonomous driving and video surveillance. However, it still remains a crucial challenge to train a reliable detector working well in different multispectral pedestrian datasets without manual annotations. In this paper, we propose a novel unsupervised domain adaptation framework for multispectral pedestrian detection, by iteratively generating pseudo annotations and updating the parameters of our designed multispectral pedestrian det…
Centro de Procesamiento Digital de Imágenes（数字图像处理中心）；Instituto de Ingeniería（工程研究所）；Laboratorio de Computación Gráfica y Geometría Aplicada（图形与应用几何实验室）；Escuela de Matemática（数学学院）；Facultad de Ciencias（理学院）；Universidad Central de Venezuela（委内瑞拉中央大学）；Centro de Computación Gráfica（图形计算中心）；Escuela de Computación（计算机学院）
Multispectral images acquired by satellites are used to study phenomena on the Earth's surface. Unsupervised classification techniques analyze multispectral image content without considering prior knowledge of the observed terrain; this is done using techniques which group pixels that have similar statistics of digital level distribution in the various image channels. In this paper, we propose a methodology for unsupervised classification based on a deterministic cellular automaton. The automaton is initialized in an unsupervised manner by setting seed cells, selected according to two criteria…
University of Poitiers（普瓦捷大学）；DGA/DS/QIS（法国国防部装备总署/情报处/量子信息部门）
We introduce a new constrained minimization problem that performs template and pattern detection on a multispectral image in a compressive sensing context. We use an original minimization problem from Guo and Osher that uses $L_1$ minimization techniques to perform template detection in a multispectral image. We first adapt this minimization problem to work with compressive sensing data. Then we extend it to perform pattern detection using a formal transform called the spectralization along a pattern. That extension brings out the problem of measurement reconstruction. We introduce shifted mea…
Unsupervised Hyperspectral and Multispectral Image Blind Fusion Based on Deep Tucker Decomposition Network with Spatial-Spectral Manifold Learning
School of Computer Science and Engineering, Nanjing University of Science and Technology (NJUST)（南京理工大学计算机科学与工程学院）；Geological Exploration Technology Institute of Jiangsu Province（江苏省地质勘查技术院）；Jiangsu Province Engineering Research Center of Airborne Detecting and Intelligent Perceptive Technology（江苏省机载探测与智能感知技术工程研究中心）
We address the problem of registering synchronized color (RGB) and multi-spectral (MS) images featuring very different resolution by solving stereo matching correspondences. Purposely, we introduce a novel RGB-MS dataset framing 13 different scenes in indoor environments and providing a total of 34 image pairs annotated with semi-dense, high-resolution ground-truth labels in the form of disparity maps. To tackle the task, we propose a deep learning architecture trained in a self-supervised manner by exploiting a further RGB camera, required only during training data acquisition. In this setup,…
Objectives. Sustainable management of plant diseases is an open challenge which has relevant economic and environmental impact. Optimal strategies rely on human expertise for field scouting under favourable conditions to assess the current presence and extent of disease symptoms. This labor-intensive task is complicated by the large field area to be scouted, combined with the millimeter-scale size of the early symptoms to be detected. In view of this, image-based detection of early disease symptoms is an attractive approach to automate this process, enabling a potential high throughput monitor…
Unsupervised semantic segmentation of urban high-density multispectral point clouds
Finnish Geospatial Research Institute FGI, the National Land Survey of Finland（芬兰地理空间研究所（FGI，芬兰国家土地测量局））；Department of Built Environment（建筑环境系）；School of Engineering（工程学院）；Aalto University（阿尔托大学）
本文提出无监督地面感知深度聚类方法 GroupSP，通过超点分组与伪标签迭代训练，实现高密度多光谱 ALS 城市点云的七类语义分割，并以极少标注取得高精度。
Unsupervised deep learning for semantic segmentation of multispectral LiDAR forest point clouds
Department of Remote Sensing and Photogrammetry, Finnish Geospatial Research Institute FGI, The National Land Survey of Finland（遥感与摄影测量系，芬兰地理研究 institute FGI，芬兰国家土地测绘局）；D Optical Metrology (3DOM) unit, Bruno Kessler Foundation (FBK)（3D光学计量（3DOM）单元，布鲁诺·凯瑟尔基金会（FBK））；Department of Built Environment, Aalto University, School of Engineering（环境学院，艾尔托大学，工程学院）
Benchmarking individual tree segmentation using multispectral airborne laser scanning data: the FGI-EMIT dataset
Finnish Geospatial Research Institute FGI（芬兰地理空间研究 institute）；The National Land Survey of Finland（芬兰国家土地调查局）；Department of Remote Sensing and Photogrammetry（遥感与摄影测量系）；Department of Built Environment, School of Engineering, Aalto University（环境学院，工程学院，阿alto大学）
Monotone Piecewise-Linear Harmonization of Multitemporal Multispectral Surface Reflectance With Unsupervised Deep Multiview Pseudoinvariant Features Learning
National Research and Innovation Agency (BRIN)（国家研究与创新署）；Department of Electrical Engineering, National Cheng Kung University (NCKU)（国立成功大学电气工程学系）
Pansharpening in remote sensing image aims at acquiring a high-resolution multispectral (HRMS) image directly by fusing a low-resolution multispectral (LRMS) image with a panchromatic (PAN) image. The main concern is how to effectively combine the rich spectral information of LRMS image with the abundant spatial information of PAN image. Recently, many methods based on deep learning have been proposed for the pansharpening task. However, these methods usually has two main drawbacks: 1) requiring HRMS for supervised learning; and 2) simply ignoring the latent relation between the MS and PAN ima…
University of Nebraska-Lincoln（内布拉斯加大学林肯分校）；University of California, Berkeley（加州大学伯克利分校）；Boston University（波士顿大学）；Lisbon University（里斯本大学）；Northwest Nazarene University（西北拿撒勒大学）；University of Oxford（牛津大学）
The accurate characterization of the severity of the wildfire event strongly contributes to the characterization of the fuel conditions in fire-prone areas, and provides valuable information for disaster response. The aim of this study is to develop an autonomous system built on top of high-resolution multispectral satellite imagery, with an advanced deep learning method for detecting burned area change. This work proposes an initial exploration of using an unsupervised model for feature extraction in wildfire scenarios. It is based on the contrastive learning technique SimCLR, which is traine…
Recently proposed DNN-based stereo matching methods that learn priors directly from data are known to suffer a drastic drop in accuracy in new environments. Although supervised approaches with ground truth disparity maps often work well, collecting them in each deployment environment is cumbersome and costly. For this reason, many unsupervised domain adaptation methods based on image-to-image translation have been proposed, but these methods do not preserve the geometric structure of a stereo image pair because the image-to-image translation is applied to each view separately. To address this …
Unlabeled data learning has attracted considerable attention recently. However, it is still elusive to extract the expected high-level semantic feature with mere unsupervised learning. In the meantime, semi-supervised learning (SSL) demonstrates a promising future in leveraging few samples. In this paper, we combine both to propose an Unsupervised Semantic Aggregation and Deformable Template Matching (USADTM) framework for SSL, which strives to improve the classification performance with few labeled data and then reduce the cost in data annotating. Specifically, unsupervised semantic aggregati…
Recently, there are emerging many stereo matching methods for autonomous driving based on unsupervised learning. Most of them take advantage of reconstruction losses to remove dependency on disparity groundtruth. Occlusion handling is a challenging problem in stereo matching, especially for unsupervised methods. Previous unsupervised methods failed to take full advantage of geometry properties in occlusion handling. In this paper, we introduce an effective way to detect occlusion regions and propose a novel unsupervised training strategy to deal with occlusion that only uses the predicted left…
At present, deep learning has been applied more and more in monocular image depth estimation and has shown promising results. The current more ideal method for monocular depth estimation is the supervised learning based on ground truth depth, but this method requires an abundance of expensive ground truth depth as the supervised labels. Therefore, researchers began to work on unsupervised depth estimation methods. Although the accuracy of unsupervised depth estimation method is still lower than that of supervised method, it is a promising research direction. In this paper, Based on the experim…
Stereo matching is a key component of autonomous driving perception. Recent unsupervised stereo matching approaches have received adequate attention due to their advantage of not requiring disparity ground truth. These approaches, however, perform poorly near occlusions. To overcome this drawback, in this paper, we propose CoT-Stereo, a novel unsupervised stereo matching approach. Specifically, we adopt a co-teaching framework where two networks interactively teach each other about the occlusions in an unsupervised fashion, which greatly improves the robustness of unsupervised stereo matching.…
Microsoft（微软）；IIT Kanpur（印度理工学院坎普尔分校）；Oracle（甲骨文公司）；University of Bath（巴斯大学）
Understanding unsupervised domain adaptation has been an important task that has been well explored. However, the wide variety of methods have not analyzed the role of a classifier's performance in detail. In this paper, we thoroughly examine the role of a classifier in terms of matching source and target distributions. We specifically investigate the classifier ability by matching a) the distribution of features, b) probabilistic uncertainty for samples and c) certainty activation mappings. Our analysis suggests that using these three distributions does result in a consistently improved perfo…
ENS（巴黎高等师范学院）；CNRS（法国国家科学研究中心）；PSL University（巴黎文理研究大学）；INRIA（法国国家信息与自动化研究所）；POSTECH（浦项科技大学）；University of Oxford（牛津大学）；New York University（纽约大学）
Learning with complete or partial supervision is powerful but relies on ever-growing human annotation efforts. As a way to mitigate this serious problem, as well as to serve specific applications, unsupervised learning has emerged as an important field of research. In computer vision, unsupervised learning comes in various guises. We focus here on the unsupervised discovery and matching of object categories among images in a collection, following the work of Cho et al. 2015. We show that the original approach can be reformulated and solved as a proper optimization problem. Experiments on sever…
Beihang University（北京航空航天大学）；SenseTime Research（商汤科技研究院）；The Chinese University of Hong Kong（香港中文大学）
Unsupervised cross-spectral stereo matching aims at recovering disparity given cross-spectral image pairs without any supervision in the form of ground truth disparity or depth. The estimated depth provides additional information complementary to individual semantic features, which can be helpful for other vision tasks such as tracking, recognition and detection. However, there are large appearance variations between images from different spectral bands, which is a challenge for cross-spectral stereo matching. Existing deep unsupervised stereo matching methods are sensitive to the appearance v…
College of Electronics and Information Engineering, Tongji University（同济大学电子信息工程学院）；Harbin Institute of Technology Suzhou Research Institute（哈尔滨工业大学苏州研究院）；Department of Computer Science and Technology, Tongji University（同济大学计算机科学与技术系）；Multimedia Technology and Telecom Department, Telecommunications Center, Moscow Institute of Physics（莫斯科物理技术学院多媒体技术与电信系、电信中心）
Qiushi Academy for Advanced Studies, Zhejiang University（浙江大学求是高等研究院）；College of Computer Science and Technology, Zhejiang University（浙江大学计算机科学与技术学院）；State University of New York at Buffalo（纽约州立大学布法罗分校）；Zhejiang University School of Medicine（浙江大学医学院）
Domain adaptation is an important technique to alleviate performance degradation caused by domain shift, e.g., when training and test data come from different domains. Most existing deep adaptation methods focus on reducing domain shift by matching marginal feature distributions through deep transformations on the input features, due to the unavailability of target domain labels. We show that domain shift may still exist via label distribution shift at the classifier, thus deteriorating model performances. To alleviate this issue, we propose an approximate joint distribution matching scheme by…
Tsinghua University（清华大学）；Beijing National Research Center for Information Science and Technologies（北京信息科学与技术国家研究中心）
Remarkable gains in deep learning usually rely on tremendous supervised data. Ensuring the modality diversity for one object in training set is critical for the generalization of cutting-edge deep models, but it burdens human with heavy manual labor on data collection and annotation. In addition, some rare or unexpected modalities are new for the current model, causing reduced performance under such emerging modalities. Inspired by the achievements in speech recognition, psychology and behavioristics, we present a practical solution, self-reinforcing unsupervised matching (SUM), to annotate th…
Hong Kong Baptist University（香港浸会大学）；University of Technology Sydney（悉尼科技大学）
Label estimation is an important component in an unsupervised person re-identification (re-ID) system. This paper focuses on cross-camera label estimation, which can be subsequently used in feature learning to learn robust re-ID models. Specifically, we propose to construct a graph for samples in each camera, and then graph matching scheme is introduced for cross-camera labeling association. While labels directly output from existing graph matching methods may be noisy and inaccurate due to significant cross-camera variations, this paper proposes a dynamic graph matching (DGM) method. DGM iter…
The Australian National University（澳大利亚国立大学）；Queensland University of Technology（昆士兰科技大学）
We propose a new task of unsupervised action detection by action matching. Given two long videos, the objective is to temporally detect all pairs of matching video segments. A pair of video segments are matched if they share the same human action. The task is category independent---it does not matter what action is being performed---and no supervision is used to discover such video segments. Unsupervised action detection by action matching allows us to align videos in a meaningful manner. As such, it can be used to discover new action categories or as an action proposal technique within, say, …
We present G-MSM (Graph-based Multi-Shape Matching), a novel unsupervised learning approach for non-rigid shape correspondence. Rather than treating a collection of input poses as an unordered set of samples, we explicitly model the underlying shape data manifold. To this end, we propose an adaptive multi-shape matching architecture that constructs an affinity graph on a given set of training shapes in a self-supervised manner. The key idea is to combine putative, pairwise correspondences by propagating maps along shortest paths in the underlying shape graph. During training, we enforce cycle-…
Progressive Pixel-Neighborhood Deformable Cross-Attention for Multispectral Object Detection
School of Electrical and Information Engineering, Jiangsu University（江苏大学电气与信息工程学院）；School of Computer Science and Engineering, Jiangsu University of Science and Technology（江苏科技大学计算机科学与工程学院）
Matching articulated shapes represented by voxel-sets reduces to maximal sub-graph isomorphism when each set is described by a weighted graph. Spectral graph theory can be used to map these graphs onto lower dimensional spaces and match shapes by aligning their embeddings in virtue of their invariance to change of pose. Classical graph isomorphism schemes relying on the ordering of the eigenvalues to align the eigenspaces fail when handling large data-sets or noisy data. We derive a new formulation that finds the best alignment between two congruent $K$-dimensional sets of points by selecting …
University of Bonn（波恩大学）；Technical University of Munich（慕尼黑工业大学）
3D shape matching is a long-standing problem in computer vision and computer graphics. While deep neural networks were shown to lead to state-of-the-art results in shape matching, existing learning-based approaches are limited in the context of multi-shape matching: (i) either they focus on matching pairs of shapes only and thus suffer from cycle-inconsistent multi-matchings, or (ii) they require an explicit template shape to address the matching of a collection of shapes. In this paper, we present a novel approach for deep multi-shape matching that ensures cycle-consistent multi-matchings whi…
The unsupervised segmentation is an increasingly popular topic in biomedical image analysis. The basic idea is to approach the supervised segmentation task as an unsupervised synthesis problem, where the intensity images can be transferred to the annotation domain using cycle-consistent adversarial learning. The previous studies have shown that the macro-level (global distribution level) matching on the number of the objects (e.g., cells, tissues, protrusions etc.) between two domains resulted in better segmentation performance. However, no prior studies have exploited whether the unsupervised…
Inria（法国国家信息与自动化研究所）；École Normale Supérieure（巴黎高等师范学院）；PSL Research University（巴黎文理研究大学）
This paper addresses unsupervised discovery and localization of dominant objects from a noisy image collection with multiple object classes. The setting of this problem is fully unsupervised, without even image-level annotations or any assumption of a single dominant class. This is far more general than typical colocalization, cosegmentation, or weakly-supervised localization tasks. We tackle the discovery and localization problem using a part-based region matching approach: We use off-the-shelf region proposals to form a set of candidate bounding boxes for objects and object parts. These regi…
Lahore University of Management Sciences（拉合尔管理科学大学）
The advent of deep perceptual networks brought about a paradigm shift in machine vision and image perception. Image apprehension lately carried out by hand-crafted features in the latent space have been replaced by deep features acquired from supervised networks for improved understanding. However, such deep networks require strict supervision with a substantial amount of the labeled data for authentic training process. These methods perform poorly in domains lacking labeled data especially in case of remote sensing image retrieval. Resolving this, we propose an unsupervised encoder-decoder fe…
Minimizing the discrepancy of feature distributions between different domains is one of the most promising directions in unsupervised domain adaptation. From the perspective of distribution matching, most existing discrepancy-based methods are designed to match the second-order or lower statistics, which however, have limited expression of statistical characteristic for non-Gaussian distributions. In this work, we explore the benefits of using higher-order statistics (mainly refer to third-order and fourth-order statistics) for domain matching. We propose a Higher-order Moment Matching (HoMM) …
We present a novel method for computing correspondences across 3D shapes using unsupervised learning. Our method computes a non-linear transformation of given descriptor functions, while optimizing for global structural properties of the resulting maps, such as their bijectivity or approximate isometry. To this end, we use the functional maps framework, and build upon the recent FMNet architecture for descriptor learning. Unlike that approach, however, we show that learning can be done in a purely \emph{unsupervised setting}, without having access to any ground truth correspondences. This resu…
We consider the task of multimodal one-shot speech-image matching. An agent is shown a picture along with a spoken word describing the object in the picture, e.g. cookie, broccoli and ice-cream. After observing one paired speech-image example per class, it is shown a new set of unseen pictures, and asked to pick the "ice-cream". Previous work attempted to tackle this problem using transfer learning: supervised models are trained on labelled background data not containing any of the one-shot classes. Here we compare transfer learning to unsupervised models trained on unlabelled in-domain data. …
Research Center for Information Technology Innovation, Academia Sinica（中央研究院资讯科技创新研究中心）；Department of Computer Science and Information Engineering, National Taiwan University（国立台湾大学资讯工程学系）
With the aim to improve the performance of feature matching, we present an unsupervised approach to fuse various local descriptors in the space of homographies. Inspired by the observation that the homographies of correct feature correspondences vary smoothly along the spatial domain, our approach stands on the unsupervised nature of feature matching, and can select a good descriptor for matching each feature point. Specifically, the homography space serves as the common domain, in which a correspondence obtained by any descriptor is considered as a point, for integrating various heterogeneous…
Hunan University（湖南大学）；National Engineering Laboratory for Robot Visual Perception and Control Technology（机器人视觉感知与控制技术国家工程实验室）；University of California, Riverside（加州大学河滨分校）
Many unsupervised approaches have been proposed recently for the video-based re-identification problem since annotations of samples across cameras are time-consuming. However, higher-order relationships across the entire camera network are ignored by these methods, leading to contradictory outputs when matching results from different camera pairs are combined. In this paper, we address the problem of unsupervised video-based re-identification by proposing a consistent cross-view matching (CCM) framework, in which global camera network constraints are exploited to guarantee the matched pairs ar…
Beijing Institute of Technology（北京理工大学）；The University of Adelaide（阿德莱德大学）；Australian Centre for Robotic Vision（澳大利亚机器人视觉中心）
We propose a fast, accurate matching method for estimating dense pixel correspondences across scenes. It is a challenging problem to estimate dense pixel correspondences between images depicting different scenes or instances of the same object category. While most such matching methods rely on hand-crafted features such as SIFT, we learn features from a large amount of unlabeled image patches using unsupervised learning. Pixel-layer features are obtained by encoding over the dictionary, followed by spatial pooling to obtain patch-layer features. The learned features are then seamlessly embedde…
LIGM (UMR 8049) École des Ponts UPE（巴黎桥路大学LIGM实验室（法国国家科研中心8049混合研究单位））；Adobe Research（奥多比研究院）
We propose a self-supervised approach to deep surface deformation. Given a pair of shapes, our algorithm directly predicts a parametric transformation from one shape to the other respecting correspondences. Our insight is to use cycle-consistency to define a notion of good correspondences in groups of objects and use it as a supervisory signal to train our network. Our method does not rely on a template, assume near isometric deformations or rely on point-correspondence supervision. We demonstrate the efficacy of our approach by using it to transfer segmentation across shapes. We show, on Shap…
Unsupervised domain adaptation seeks to mitigate the distribution discrepancy between source and target domains, given labeled samples of the source domain and unlabeled samples of the target domain. Generative adversarial networks (GANs) have demonstrated significant improvement in domain adaptation by producing images which are domain specific for training. However, most of the existing GAN based techniques for unsupervised domain adaptation do not consider semantic information during domain matching, hence these methods degrade the performance when the source and target domain data are sema…
