Xidian University（西安电子科技大学）；Indiana University-Purdue University Indianapolis（印第安纳大学与普渡大学印第安纳波利斯联合分校）；Northeastern University（东北大学）
Qianqian Wang, Zhengming Ding, Zhiqiang Tao, Quanxue Gao, Yun Fu
备注：
This paper is an extension to our previous work: "Wang Q, Ding Z, Tao Z, et al. Partial multi-view clustering via consistent GAN[C]//2018 IEEE International Conference on Data Mining (ICDM). IEEE, 2018: 1290-1295."
Nowadays, with the rapid development of data collection sources and feature extraction methods, multi-view data are getting easy to obtain and have received increasing research attention in recent years, among which, multi-view clustering (MVC) forms a mainstream research direction and is widely used in data analysis. However, existing MVC methods mainly assume that each sample appears in all the views, without considering the incomplete view case due to data corruption, sensor failure, equipment malfunction, etc. In this study, we design and build a generative partial multi-view clustering model, named as GP-MVC, to address the incomplete multi-view problem by explicitly generating the data of missing views. The main idea of GP-MVC lies at two-fold. First, multi-view encoder networks are trained to learn common low-dimensional representations, followed by a clustering layer to capture the consistent cluster structure across multiple views. Second, view-specific generative adversarial networks are developed to generate the missing data of one view conditioning on the shared representation given by other views. These two steps could be promoted mutually, where learning common representations facilitates data imputation and the generated data could further explores the view consistency. Moreover, an weighted adaptive fusion scheme is implemented to exploit the complementary information among different views. Experimental results on four benchmark datasets are provided to show the effectiveness of the proposed GP-MVC over the state-of-the-art methods.
Unsupervised fine-grained class clustering is a practical yet challenging task due to the difficulty of feature representations learning of subtle object details. We introduce C3-GAN, a method that leverages the categorical inference power of InfoGAN with contrastive learning. We aim to learn feature representations that encourage a dataset to form distinct cluster boundaries in the embedding space, while also maximizing the mutual information between the latent code and its image observation. Our approach is to train a discriminator, which is also used for inferring clusters, to optimize the …
State Key Laboratory of Robotics, Shenyang Institute of Automation, Chinese Academy of Sciences（机器人学国家重点实验室，中国科学院沈阳自动化研究所）；University of Chinese Academy of Sciences（中国科学院大学）
Visual-tactile fused sensing for object clustering has achieved significant progresses recently, since the involvement of tactile modality can effectively improve clustering performance. However, the missing data (i.e., partial data) issues always happen due to occlusion and noises during the data collecting process. This issue is not well solved by most existing partial multi-view clustering methods for the heterogeneous modality challenge. Naively employing these methods would inevitably induce a negative effect and further hurt the performance. To solve the mentioned challenges, we propose …
Xiaomi AI Lab（小米人工智能实验室）；Chinese University of Hong Kong（香港中文大学）；Peking University（北京大学）
Training generative models like Generative Adversarial Network (GAN) is challenging for noisy data. A novel curriculum learning algorithm pertaining to clustering is proposed to address this issue in this paper. The curriculum construction is based on the centrality of underlying clusters in data points. The data points of high centrality takes priority of being fed into generative models during training. To make our algorithm scalable to large-scale data, the active set is devised, in the sense that every round of training proceeds only on an active subset containing a small fraction of alrea…
Tokyo University of Agriculture and Technology（东京农工大学）
A yuru-chara is a mascot character created by local governments and companies for publicizing information on areas and products. Because it takes various costs to create a yuruchara, the utilization of machine learning techniques such as generative adversarial networks (GANs) can be expected. In recent years, it has been reported that the use of class conditions in a dataset for GANs training stabilizes learning and improves the quality of the generated images. However, it is difficult to apply class conditional GANs when the amount of original data is small and when a clear class is not given…
Straight-Path Flow Matching for Incomplete Multi-View Clustering
School of Software Engineering, Huazhong University of Science and Technology（华中科技大学软件工程学院）；Australian Institute for Machine Learning, Adelaide University（澳大利亚机器学习研究所，阿德莱德大学）；University of Auckland（奥克兰大学）；University of New South Wales（新南威尔士大学）；Communication University of China（中国通信大学）
Several deep learning methods have been proposed for completing partial data from shape acquisition setups, i.e., filling the regions that were missing in the shape. These methods, however, only complete the partial shape with a single output, ignoring the ambiguity when reasoning the missing geometry. Hence, we pose a multi-modal shape completion problem, in which we seek to complete the partial shape with multiple outputs by learning a one-to-many mapping. We develop the first multimodal shape completion method that completes the partial shape via conditional generative modeling, without req…
Institute for Informatics（信息学研究所）；LMU Munich（慕尼黑大学）
Over the past years, Generative Adversarial Networks (GANs) have shown a remarkable generation performance especially in image synthesis. Unfortunately, they are also known for having an unstable training process and might loose parts of the data distribution for heterogeneous input data. In this paper, we propose a novel GAN extension for multi-modal distribution learning (MMGAN). In our approach, we model the latent space as a Gaussian mixture model with a number of clusters referring to the number of disconnected data manifolds in the observation space, and include a clustering network, whi…
Keele University（基尔大学）；Indian Institute of Technology, Bhubaneswar（印度理工学院布巴内斯瓦尔分校）
Traditional machine learning algorithms using hand-crafted feature extraction techniques (such as local binary pattern) have limited accuracy because of high variation in images of the same class (or intra-class variation) for food recognition task. In recent works, convolutional neural networks (CNN) have been applied to this task with better results than all previously reported methods. However, they perform best when trained with large amount of annotated (labeled) food images. This is problematic when obtained in large volume, because they are expensive, laborious and impractical. Our work…
While GAN is a powerful model for generating images, its inability to infer a latent space directly limits its use in applications requiring an encoder. Our paper presents a simple architectural setup that combines the generative capabilities of GAN with an encoder. We accomplish this by combining the encoder with the discriminator using shared weights, then training them simultaneously using a new loss term. We model the output of the encoder latent space via a GMM, which leads to both good clustering using this latent space and improved image generation by the GAN. Our framework is generic a…
Action Prediction is aimed to determine what action is occurring in a video as early as possible, which is crucial to many online applications, such as predicting a traffic accident before it happens and detecting malicious actions in the monitoring system. In this work, we address this problem by developing an end-to-end architecture that improves the discriminability of features of partially observed videos by assimilating them to features from complete videos. For this purpose, the generative adversarial network is introduced for tackling action prediction problem, which improves the recogn…
Conditional generative models have achieved considerable success in the past few years, but usually require a lot of labeled data. Recently, ClusterGAN combines GAN with an encoder to achieve remarkable clustering performance via unsupervised conditional generation. However, it ignores the real conditional distribution of data, which leads to generating less diverse samples for each class and makes the encoder only achieve sub-optimal clustering performance. Here, we propose a new unsupervised conditional generation framework, Double Cycle-Consistent Conditional GAN (DC3-GAN), which can genera…
Generative Adversarial Networks (GANs) have been shown to produce realistically looking synthetic images with remarkable success, yet their performance seems less impressive when the training set is highly diverse. In order to provide a better fit to the target data distribution when the dataset includes many different classes, we propose a variant of the basic GAN model, called Gaussian Mixture GAN (GM-GAN), where the probability distribution over the latent space is a mixture of Gaussians. We also propose a supervised variant which is capable of conditional sample synthesis. In order to eval…
Department of Computer Science, Ben-Gurion University of the Negev, Beer Sheva, Israel（计算机科学系，贝尔谢巴以色列内盖夫比尔亚尔大学）；Department of Industrial Engineering and Management, Ben-Gurion University of the Negev, Beer Sheva, Israel（工业工程与管理系，贝尔谢巴以色列内盖夫比尔亚尔大学）；The School of Brain Sciences and Cognition, Ben-Gurion University of the Negev, Beer Sheva, Israel（脑科学与认知学院，贝尔谢巴以色列内盖夫比尔亚尔大学）；Data Science Research Center, Ben-Gurion University of the Negev, Beer Sheva, Israel（数据科学研究中心，贝尔谢巴以色列内盖夫比尔亚尔大学）
ETH Zurich（苏黎世联邦理工学院）；Merantix GmbH（Merantix 有限公司）；KU Leuven（鲁汶大学）
Designing a logo for a new brand is a lengthy and tedious back-and-forth process between a designer and a client. In this paper we explore to what extent machine learning can solve the creative task of the designer. For this, we build a dataset -- LLD -- of 600k+ logos crawled from the world wide web. Training Generative Adversarial Networks (GANs) for logo synthesis on such multi-modal data is not straightforward and results in mode collapse for some state-of-the-art methods. We propose the use of synthetic labels obtained through clustering to disentangle and stabilize GAN training. We are a…
We propose the use of unsupervised learning to train projection networks that project onto the latent space of an already trained generator. We apply our method to a trained StyleGAN, and use our projection network to perform image super-resolution and clustering of images into semantically identifiable groups.
Beijing Institute of Technology（北京理工大学）；Tencent AI Lab（腾讯AI实验室）；Hulu LLC.（Hulu有限责任公司）
Clustering is among the most fundamental tasks in computer vision and machine learning. In this paper, we propose Variational Deep Embedding (VaDE), a novel unsupervised generative clustering approach within the framework of Variational Auto-Encoder (VAE). Specifically, VaDE models the data generative procedure with a Gaussian Mixture Model (GMM) and a deep neural network (DNN): 1) the GMM picks a cluster; 2) from which a latent embedding is generated; 3) then the DNN decodes the latent embedding into observables. Inference in VaDE is done in a variational way: a different DNN is used to encod…
Generative Diffusion Contrastive Network for Multi-View Clustering
Zhejiang Lab（浙江实验室）；Hong Kong University of Science and Technology（香港科技大学）；University of Science and Technology of China（中国科学技术大学）；Huazhong University of Science and Technology（华中科技大学）
Tokyo University of Agriculture and Technology（东京农工大学）
Toward a computer-assisted marking for descriptive math questions,this paper presents clustering of online handwritten mathematical expressions (OnHMEs) to help human markers to mark them efficiently and reliably. We propose a generative sequence similarity function for computing a similarity score of two OnHMEs based on a sequence-to-sequence OnHME recognizer. Each OnHME is represented by a similarity-based representation (SbR) vector. The SbR matrix is inputted to the k-means algorithm for clustering OnHMEs. Experiments are conducted on an answer dataset (Dset_Mix) of 200 OnHMEs mixed of rea…
Guangdong University of Technology（广东工业大学）；The University of Sydney（悉尼大学）
Clustering multi-view data has been a fundamental research topic in the computer vision community. It has been shown that a better accuracy can be achieved by integrating information of all the views than just using one view individually. However, the existing methods often struggle with the issues of dealing with the large-scale datasets and the poor performance in reconstructing samples. This paper proposes a novel multi-view clustering method by learning a shared generative latent representation that obeys a mixture of Gaussian distributions. The motivation is based on the fact that the mul…
Generalized Deep Multi-view Clustering via Causal Learning with Partially Aligned Cross-view Correspondence
College of Computer Science and Technology, National University of Defense Technology（计算机科学与技术学院，国防科技大学）；Intelligent Game and Decision Lab（智能游戏与决策实验室）；National University of Singapore（新加坡国立大学）；Chinese PLA General Hospital（中国人民解放军总医院）
Reducing Bias and Variance: Generative Semantic Guidance and Bi-Layer Ensemble for Image Clustering
Institute of Big Data Science and Industry（大数据科学与产业研究院）；Key Laboratory of Evolutionary Science Intelligence of Shanxi Province（山西省进化智能科学重点实验室）；School of Artificial Intelligence, Shanxi University（山西大学人工智能学院）
In this paper, we propose an augmentation-free graph contrastive learning framework, namely ACTIVE, to solve the problem of partial multi-view clustering. Notably, we suppose that the representations of similar samples (i.e., belonging to the same cluster) and their multiply views features should be similar. This is distinct from the general unsupervised contrastive learning that assumes an image and its augmentations share a similar representation. Specifically, relation graphs are constructed using the nearest neighbours to identify existing similar samples, then the constructed inter-instan…
Beijing University of Posts and Telecommunications（北京邮电大学）
Subspace clustering refers to the problem of segmenting high dimensional data drawn from a union of subspaces into the respective subspaces. In some applications, partial side-information to indicate "must-link" or "cannot-link" in clustering is available. This leads to the task of subspace clustering with side-information. However, in prior work the supervision value of the side-information for subspace clustering has not been fully exploited. To this end, in this paper, we present an enhanced approach for constrained subspace clustering with side-information, termed Constrained Sparse Subspa…
Robust 3D Alignment of Generative Reconstructions via Partial Monocular Observations
School of Computer Science, Shanghai Jiao Tong University（上海交通大学计算机科学与技术学院）；School of Computer Science and Engineering, Nanjing University of Science and Technology（南京理工大学计算机科学与工程学院）；School of Engineering and Applied Science, University of Pennsylvania（宾夕法尼亚大学工程与应用科学学院）；School of Tech Electrical and Information Engineering, Tianjin University（天津大学电气与信息工程学院）；Independent Researcher（独立研究员）；School of Electronic Engineering and Information Science, University of Science and Technology of China（中国科学技术大学电子工程与信息科学学院）；Gaoling School of Artificial Intelligence, Renmin University of China（中国人民大学高瓴人工智能学院）；School of Robotics, Wuhan University（武汉大学机器人学院）；Xiaomi EV（小米汽车）
The success of existing multi-view clustering relies on the assumption of sample integrity across multiple views. However, in real-world scenarios, samples of multi-view are partially available due to data corruption or sensor failure, which leads to incomplete multi-view clustering study (IMVC). Although several attempts have been proposed to address IMVC, they suffer from the following drawbacks: i) Existing methods mainly adopt cross-view contrastive learning forcing the representations of each sample across views to be exactly the same, which might ignore view discrepancy and flexibility i…
Deep neural network models trained on large labeled datasets are the state-of-the-art in a large variety of computer vision tasks. In many applications, however, labeled data is expensive to obtain or requires a time consuming manual annotation process. In contrast, unlabeled data is often abundant and available in large quantities. We present a principled framework to capitalize on unlabeled data by training deep generative models on both labeled and unlabeled data. We show that such a combination is beneficial because the unlabeled data acts as a data-driven form of regularization, allowing …
Vidyasirimedhi Institute of Science and Technology (VISTEC)（维迪亚西利梅迪科学技术学院）；King Mongkut’s University of Technology Thonburi (KMUTT)（国王科技大学吞武里分校）；Imperial College London（帝国理工学院）
Visual data can be understood at different levels of granularity, where global features correspond to semantic-level information and local features correspond to texture patterns. In this work, we propose a framework, called SPLIT, which allows us to disentangle local and global information into two separate sets of latent variables within the variational autoencoder (VAE) framework. Our framework adds generative assumption to the VAE by requiring a subset of the latent variables to generate an auxiliary set of observable data. This additional generative assumption primes the latent variables …
DAS-PMVC: A Framework for Partial Multi-View Clustering via Dual Alignment and Structure Enhancement
Dalian University of Technology（大连理工大学）；Inspur Group Co., Ltd.（浪潮集团有限公司）；China University of Mining and Technology（中国矿业大学）；The University of Warwick（华威大学）
Self-supervised learning is a popular and powerful method for utilizing large amounts of unlabeled data, for which a wide variety of training objectives have been proposed in the literature. In this study, we perform a Bayesian analysis of state-of-the-art self-supervised learning objectives and propose a unified formulation based on likelihood learning. Our analysis suggests a simple method for integrating self-supervised learning with generative models, allowing for the joint training of these two seemingly distinct approaches. We refer to this combined framework as GEDI, which stands for GE…
SMART: Semantic Matching Contrastive Learning for Partially View-Aligned Clustering
College of Computer Science and Technology, Huaqiao University（华侨大学计算机科学与技术学院）；Department of Computer Science, Shantou University（汕头大学计算机科学系）；Chongqing Key Laboratory of Nonlinear Circuits and Intelligent Information Processing, College of Electronic and Information Engineering, Southwest University（西南大学电子信息工程学院非线性电路与智能信息处理重点实验室）；School of Computer Science and Engineering, South China University of Technology（华南理工大学计算机科学与工程学院）；Department of Computer Science, City University of Hong Kong（香港城市大学计算机科学系）
University of California, San Diego（加利福尼亚大学圣迭戈分校）；City University of Hong Kong（香港城市大学）
The hidden Markov model (HMM) is a widely-used generative model that copes with sequential data, assuming that each observation is conditioned on the state of a hidden Markov chain. In this paper, we derive a novel algorithm to cluster HMMs based on the hierarchical EM (HEM) algorithm. The proposed algorithm i) clusters a given collection of HMMs into groups of HMMs that are similar, in terms of the distributions they represent, and ii) characterizes each group by a "cluster center", i.e., a novel HMM that is representative for the group, in a manner that is consistent with the underlying gene…
