Clustering is a fundamental task in unsupervised learning that depends heavily on the data representation that is used. Deep generative models have appeared as a promising tool to learn informative low-dimensional data representations. We propose Matching Priors and Conditionals for Clustering (MPCC), a GAN-based model with an encoder to infer latent variables and cluster categories from data, and a flexible decoder to generate samples from a conditional latent space. With MPCC we demonstrate that a deep generative model can be competitive/superior against discriminative methods in clustering tasks surpassing the state of the art over a diverse set of benchmark datasets. Our experiments show that adding a learnable prior and augmenting the number of encoder updates improve the quality of the generated samples, obtaining an inception score of 9.49 $\pm$ 0.15 and improving the Fréchet inception distance over the state of the art by a 46.9% in CIFAR10.
Tokyo University of Agriculture and Technology（东京农工大学）
A yuru-chara is a mascot character created by local governments and companies for publicizing information on areas and products. Because it takes various costs to create a yuruchara, the utilization of machine learning techniques such as generative adversarial networks (GANs) can be expected. In recent years, it has been reported that the use of class conditions in a dataset for GANs training stabilizes learning and improves the quality of the generated images. However, it is difficult to apply class conditional GANs when the amount of original data is small and when a clear class is not given…
Conditional generative models have achieved considerable success in the past few years, but usually require a lot of labeled data. Recently, ClusterGAN combines GAN with an encoder to achieve remarkable clustering performance via unsupervised conditional generation. However, it ignores the real conditional distribution of data, which leads to generating less diverse samples for each class and makes the encoder only achieve sub-optimal clustering performance. Here, we propose a new unsupervised conditional generation framework, Double Cycle-Consistent Conditional GAN (DC3-GAN), which can genera…
CTG-KrEW: Generating Synthetic Structured Contextually Correlated Content by Conditional Tabular GAN with K-Means Clustering and Efficient Word Embedding
Indian Institute of Technology Kharagpur（印度理工学院克勒格布尔分校）；Missouri University of Science and Technology（密苏里科技大学）
GO-Based Clustering for Learning Cluster-Level Causal Gene Regulatory Networks
L3S Research Center, Leibniz University Hannover（莱布尼茨汉诺威大学L3S研究中心）；Lower Saxony Center for Artificial Intelligence and Causal Methods in Medicine (CAIMed)（下萨克森州人工智能与因果医学方法中心（CAIMed））；Institute for Causal and Explainable Artificial Intelligence in Life Sciences（生命科学因果与可解释人工智能研究所）；Center for Computational Life Sciences（计算生命科学中心）；RWTH Aachen University（亚琛工业大学）
MIT CSAIL（麻省理工学院计算机科学与人工智能实验室）；Adobe Research（Adobe研究）
We introduce a simple but effective unsupervised method for generating realistic and diverse images. We train a class-conditional GAN model without using manually annotated class labels. Instead, our model is conditional on labels automatically derived from clustering in the discriminator's feature space. Our clustering step automatically discovers diverse modes, and explicitly requires the generator to cover them. Experiments on standard mode collapse benchmarks show that our method outperforms several competing methods when addressing mode collapse. Our method also performs well on large-sca…
ETH Zürich（苏黎世联邦理工学院）；University Children’s Hospital Regensburg (KUNO)（雷根斯堡大学儿童医院（KUNO））；University of Regensburg（雷根斯堡大学）
Constrained clustering has gained significant attention in the field of machine learning as it can leverage prior information on a growing amount of only partially labeled data. Following recent advances in deep generative models, we propose a novel framework for constrained clustering that is intuitive, interpretable, and can be trained efficiently in the framework of stochastic gradient variational inference. By explicitly integrating domain knowledge in the form of probabilistic relations, our proposed model (DC-GMM) uncovers the underlying distribution of data conditioned on prior clusteri…
Vector Institute for Artificial Intelligence（向量人工智能研究院）；University of Toronto（多伦多大学）
In this paper, we describe the "implicit autoencoder" (IAE), a generative autoencoder in which both the generative path and the recognition path are parametrized by implicit distributions. We use two generative adversarial networks to define the reconstruction and the regularization cost functions of the implicit autoencoder, and derive the learning rules based on maximum-likelihood learning. Using implicit distributions allows us to learn more expressive posterior and conditional likelihood distributions for the autoencoder. Learning an expressive conditional likelihood distribution enables t…
EDF R&D（法国电力集团研发部）；Laboratoire de mathématiques d’Orsay（奥赛数学实验室）；Université Paris-Sud（巴黎南大学）；CNRS（法国国家科学研究中心）；Université Paris-Saclay（巴黎萨克雷大学）；INRIA（法国国家信息与自动化研究所）；Département d’Informatique de l’École Normale Supérieure（巴黎高等师范学院计算机系）；PSL Research University（巴黎文理研究大学）；INESC Technology and Science (INESC TEC)（INESC科技与科学研究所（INESC TEC））
The implementation of efficient demand response (DR) programs for household electricity consumption would benefit from data-driven methods capable of simulating the impact of different tariffs schemes. This paper proposes a novel method based on conditional variational autoencoders (CVAE) to generate, from an electricity tariff profile combined with exogenous weather and calendar variables, daily consumption profiles of consumers segmented in different clusters. First, a large set of consumers is gathered into clusters according to their consumption behavior and price-responsiveness. The clust…
Variational autoencoders (VAEs) have been shown to be able to generate game levels but require manual exploration of the learned latent space to generate outputs with desired attributes. While conditional VAEs address this by allowing generation to be conditioned on labels, such labels have to be provided during training and thus require prior knowledge which may not always be available. In this paper, we apply Gaussian Mixture VAEs (GMVAEs), a variant of the VAE which imposes a mixture of Gaussians (GM) on the latent space, unlike regular VAEs which impose a unimodal Gaussian. This allows GMV…
Probabilistic data quality assessment for structural monitoring data via outlier-resistant conditional diffusion model
Key Lab of Smart Prevention and Mitigation of Civil Engineering Disasters of the Ministry of Industry and Information Technology（工业和信息化部智能防灾减灾重点实验室）；School of Civil Engineering, Harbin Institute of Technology（哈尔滨工业大学土木工程学院）；Key Lab of Structures Dynamic Behavior and Control of the Ministry of Education（教育部结构动力行为与控制重点实验室）；Harbin Institute of Technology（哈尔滨工业大学）
DeepC4: Deep Conditional Census-Constrained Clustering for Large-scale Multitask Spatial Disaggregation of Urban Morphology
Department of Architecture, University of Cambridge（剑桥大学建筑系）；Cambridge University Centre for Risk in the Built Environment（剑桥大学建筑环境风险研究中心）；Earth Observation Center, German Aerospace Center（德国航天中心地球观测中心）；Institute of Geography, University of Bonn（波恩大学地理研究所）
Robust State-Conditional Feature-Weighted Jump Models for Temporal Clustering
University of Milan Department of Economics Management and Quantitative Methods（米兰大学经济学管理与定量方法系）；University of Rome "Tor Vergata" Department of Enterprise Engineering（罗马"托尔维加塔"大学企业工程系）；National Research Council of Italy Institute for Applied Mathematics and Information Technologies（意大利国家研究委员会应用数学与信息技术研究所）
Laboratoire Informatique de Paris-Nord（巴黎北部信息学实验室）；Université Sorbonne Paris Nord（索邦巴黎北大学）；Groupe Renault SAS（雷诺集团）
Autonomous driving systems validation remains one of the biggest challenges car manufacturers must tackle in order to provide safe driverless cars. The high complexity stems from several factors: the multiplicity of vehicles, embedded systems, use cases, and the very high required level of reliability for the driving system to be at least as safe as a human driver. In order to circumvent these issues, large scale simulations reproducing this huge variety of physical conditions are intensively used to test driverless cars. Therefore, the validation step produces a massive amount of data, includ…
Ludwig-Maximilians-Universität München（慕尼黑大学）；Munich Center for Machine Learning（慕尼黑机器学习中心）
A clustering outcome for high-dimensional data is typically interpreted via post-processing, involving dimension reduction and subsequent visualization. This destroys the meaning of the data and obfuscates interpretations. We propose algorithm-agnostic interpretation methods to explain clustering outcomes in reduced dimensions while preserving the integrity of the data. The permutation feature importance for clustering represents a general framework based on shuffling feature values and measuring changes in cluster assignments through custom score functions. The individual conditional expectat…
Nanyang Technological University（南洋理工大学）；DSO National Laboratories（国防科技组织国家实验室）；Macquarie University（麦考瑞大学）
Clustering evaluation measures are frequently used to evaluate the performance of algorithms. However, most measures are not properly normalized and ignore some information in the inherent structure of clusterings. We model the relation between two clusterings as a bipartite graph and propose a general component-based decomposition formula based on the components of the graph. Most existing measures are examples of this formula. In order to satisfy consistency in the component, we further propose a split-merge framework for comparing clusterings of different data sets. Our framework gives meas…
University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；Georgia Institute of Technology（佐治亚理工学院）；KAIST（韩国科学技术院）；Emory University（埃默里大学）
Graph Neural Networks (GNNs) are de facto node classification models in graph structured data. However, during testing-time, these algorithms assume no data shift, i.e., $\Pr_\text{train}(X,Y) = \Pr_\text{test}(X,Y)$. Domain adaption methods can be adopted for data shift, yet most of them are designed to only encourage similar feature distribution between source and target data. Conditional shift on classes can still affect such adaption. Fortunately, graph yields graph homophily across different data distributions. In response, we propose Shift-Robust Node Classification (SRNC) to address the…
The ability to estimate joint, conditional and marginal probability distributions over some set of variables is of great utility for many common machine learning tasks. However, estimating these distributions can be challenging, particularly in the case of data containing a mix of discrete and continuous variables. This paper presents a non-parametric method for estimating these distributions directly from a dataset. The data are first represented as a graph consisting of object nodes and attribute value nodes. Depending on the distribution to be estimated, an appropriate eigenvector equation …
Turtle shell clustering: A mixture approach to discriminative clustering with applications to flow cytometry and other data
Department of Mathematics & Statistics, McMaster University（数学与统计学系，麦斯特大学）；School of Computer Science & Statistics, Trinity College Dublin（计算机科学与统计学学院，三一学院都柏林）
Semantic Purification for Conditional Representation Learning
School of Computer Science and Engineering, Southeast University, China（计算机科学与工程学院，东南大学）；Biomedicine Discovery Institute, Department of Biochemistry and Molecular Biology, Monash University, Australia（生物医学发现研究所，墨尔本大学）
The Adversarially Learned Mixture Model (AMM) is a generative model for unsupervised or semi-supervised data clustering. The AMM is the first adversarially optimized method to model the conditional dependence between inferred continuous and categorical latent variables. Experiments on the MNIST and SVHN datasets show that the AMM allows for semantic separation of complex data when little or no labeled data is available. The AMM achieves a state-of-the-art unsupervised clustering error rate of 2.86% on the MNIST dataset. A semi-supervised extension of the AMM yields competitive results on the S…
Isolating individual instruments in a musical mixture has a myriad of potential applications, and seems imminently achievable given the levels of performance reached by recent deep learning methods. While most musical source separation techniques learn an independent model for each instrument, we propose using a common embedding space for the time-frequency bins of all instruments in a mixture inspired by deep clustering and deep attractor networks. Additionally, an auxiliary network is used to generate parameters of a Gaussian mixture model (GMM) where the posterior distribution over GMM comp…
The Conditional Cauchy-Schwarz Divergence with Applications to Time-Series Data and Sequential Decision Making
UiT - The Arctic University of Norway（挪威北极大学）；Vrije Universiteit Amsterdam（阿姆斯特丹自由大学）；University of Florida（佛罗里达大学）；NORCE Norwegian Research Centre（NORCE挪威研究中心）；Copenhagen University（哥本哈根大学）；Norwegian Computing Center（挪威计算中心）
Generative models for graphs have been typically committed to strong prior assumptions concerning the form of the modeled distributions. Moreover, the vast majority of currently available models are either only suitable for characterizing some particular network properties (such as degree distribution or clustering coefficient), or they are aimed at estimating joint probability distributions, which is often intractable in large-scale networks. In this paper, we first propose a novel network statistic, based on the Laplacian spectrum of graphs, which allows to dispense with any parametric assum…
The information bottleneck (IB) approach to clustering takes a joint distribution $P\!\left(X,Y\right)$ and maps the data $X$ to cluster labels $T$ which retain maximal information about $Y$ (Tishby et al., 1999). This objective results in an algorithm that clusters data points based upon the similarity of their conditional distributions $P\!\left(Y\mid X\right)$. This is in contrast to classic "geometric clustering'' algorithms such as $k$-means and gaussian mixture models (GMMs) which take a set of observed data points $\left\{ \mathbf{x}_{i}\right\} _{i=1:N}$ and cluster them based upon the…
Consensus Clustering of Free-Viewing Gaze Data: New Insights into Human-Information Interaction
Graphics-Visualization-Computing Lab, International Institute of Information Technology Bangalore（图形-可视化-计算实验室，国际信息技术研究所班加罗尔）；PSG College of Technology（PSG理工学院）
Massachusetts Institute of Technology（麻省理工学院）；NSF AI Institute for Artificial Intelligence and Fundamental Interactions（美国国家科学基金会人工智能与基本相互作用人工智能研究所）；University of Regina（里贾纳大学）
Anomaly Detection is becoming increasingly popular within the experimental physics community. At experiments such as the Large Hadron Collider, anomaly detection is at the forefront of finding new physics beyond the Standard Model. This paper details the implementation of a novel Machine Learning architecture, called Flux+Mutability, which combines cutting-edge conditional generative models with clustering algorithms. In the `flux' stage we learn the distribution of a reference class. The `mutability' stage at inference addresses if data significantly deviates from the reference class. We demo…
