The problem of missing data has been persistent for a long time and poses a major obstacle in machine learning and statistical data analysis. Past works in this field have tried using various data imputation techniques to fill in the missing data, or training neural networks (NNs) with the missing data. In this work, we propose a simple yet effective approach that clusters similar input features together using hierarchical clustering and then trains proportionately split neural networks with a joint loss. We evaluate this approach on a series of benchmark datasets and show promising improvements even with simple imputation techniques. We attribute this to learning through clusters of similar features in our model architecture. The source code is available at https://github.com/usarawgi911/Robustness-to-Missing-Features
Indian Statistical Institute（印度统计研究所）；Jadavpur University（贾达普大学）
Many real-world clustering problems are plagued by incomplete data characterized by missing or absent features for some or all of the data instances. Traditional clustering methods cannot be directly applied to such data without preprocessing by imputation or marginalization techniques. In this article, we overcome this drawback by utilizing a penalized dissimilarity measure which we refer to as the Feature Weighted Penalty based Dissimilarity (FWPD). Using the FWPD measure, we modify the traditional k-means clustering algorithm and the standard hierarchical agglomerative clustering algorithms…
Tennessee State University（田纳西州立大学）；Old Dominion University（老道明大学）
Missing values in tabular data restrict the use and performance of machine learning, requiring the imputation of missing values. The most popular imputation algorithm is arguably multiple imputations using chains of equations (MICE), which estimates missing values from linear conditioning on observed values. This paper proposes methods to improve both the imputation accuracy of MICE and the classification accuracy of imputed data by replacing MICE's linear conditioning with ensemble learning and deep neural networks (DNN). The imputation accuracy is further improved by characterizing individua…
Texas A&M University（得克萨斯农工大学）；TEES-AgriLife Center for Bioinformatics & Genomic Systems Engineering（TEES-农业生命中心生物信息学与基因组系统工程中心）
Missing values frequently arise in modern biomedical studies due to various reasons, including missing tests or complex profiling technologies for different omics measurements. Missing values can complicate the application of clustering algorithms, whose goals are to group points based on some similarity criterion. A common practice for dealing with missing values in the context of clustering is to first impute the missing values, and then apply the clustering algorithm on the completed data. We consider missing values in the context of optimal clustering, which finds an optimal clustering ope…
The analysis of large datasets is often complicated by the presence of missing entries, mainly because most of the current machine learning algorithms are designed to work with full data. The main focus of this work is to introduce a clustering algorithm, that will provide good clustering even in the presence of missing data. The proposed technique solves an $\ell_0$ fusion penalty based optimization problem to recover the clusters. We theoretically analyze the conditions needed for the successful recovery of the clusters. We also propose an algorithm to solve a relaxation of this problem usin…
Università degli Studi di Perugia（佩鲁贾大学）；University College Dublin（都柏林大学学院）
Gaussian Mixture models (GMMs) are a powerful tool for clustering, classification and density estimation when clustering structures are embedded in the data. The presence of missing values can largely impact the GMMs estimation process, thus handling missing data turns out to be a crucial point in clustering, classification and density estimation. Several techniques have been developed to impute the missing values before model estimation. Among these, multiple imputation is a simple and useful general approach to handle missing data. In this paper we propose two different methods to fit Gaussi…
In this paper we present deterministic conditions for success of sparse subspace clustering (SSC) under missing data, when data is assumed to come from a Union of Subspaces (UoS) model. We consider two algorithms, which are variants of SSC with entry-wise zero-filling that differ in terms of the optimization problems used to find affinity matrix for spectral clustering. For both the algorithms, we provide deterministic conditions for any pattern of missing data such that perfect clustering can be achieved. We provide extensive sets of simulation results for clustering as well as completion of …
The presence of missing entries in data often creates challenges for pattern recognition algorithms. Traditional algorithms for clustering data assume that all the feature values are known for every data point. We propose a method to cluster data in the presence of missing information. Unlike conventional clustering techniques where every feature is known for each point, our algorithm can handle cases where a few feature values are unknown for every point. For this more challenging problem, we provide theoretical guarantees for clustering using a $\ell_0$ fusion penalty based optimization prob…
A Comparison of Clustering and Missing Data Methods for Health Sciences
Institute of Mathematical Sciences（数学科学研究所）；Claremont Graduate University（克莱蒙特研究生大学）；Department of Mathematics（数学系）；Computer Science（计算机科学）；Claremont McKenna College（克莱蒙特麦肯纳学院）；School of Community（社区学院）；Global Health（全球健康学院）
Missing Pattern Tree based Decision Grouping and Ensemble for Enhancing Pair Utilization in Deep Incomplete Multi-View Clustering
School of Computer Science and Engineering, University of Electronic Science and Technology of China（电子科技大学计算机科学与工程学院）；Department of Computer Science and Engineering, Lehigh University（莱斯大学计算机科学与工程系）；Department of Computer Science, University of Illinois Chicago（伊利诺伊大学芝加哥分校计算机科学系）；School of Computer Science and Technology, Hainan University（海南大学计算机科学与技术学院）
Sparse Subspace Clustering (SSC) is a popular unsupervised machine learning method for clustering data lying close to an unknown union of low-dimensional linear subspaces; a problem with numerous applications in pattern recognition and computer vision. Even though the behavior of SSC for complete data is by now well-understood, little is known about its theoretical properties when applied to data with missing entries. In this paper we give theoretical guarantees for SSC with incomplete data, and analytically establish that projecting the zero-filled data onto the observation pattern of the poi…
In this paper, we consider the problem of multi-view clustering on incomplete views. Compared with complete multi-view clustering, the view-missing problem increases the difficulty of learning common representations from different views. To address the challenge, we propose a novel incomplete multi-view clustering framework, which incorporates cross-view relation transfer and multi-view fusion learning. Specifically, based on the consistency existing in multi-view data, we devise a cross-view relation transfer-based completion module, which transfers known similar inter-instance relationships …
A Self-Supervised Learning-based Approach to Clustering Multivariate Time-Series Data with Missing Values (SLAC-Time): An Application to TBI Phenotyping
University of Arizona（亚利桑那大学）；University of Cincinnati（辛辛那提大学）；Virginia Tech（弗吉尼亚理工大学）
Partially recorded data are frequently encountered in many applications and usually clustered by first removing incomplete cases or features with missing values, or by imputing missing values, followed by application of a clustering algorithm to the resulting altered dataset. Here, we develop clustering methodology through a model-based approach using the marginal density for the observed values, assuming a finite mixture model of multivariate $t$ distributions. We compare our approximate algorithm to the corresponding full expectation-maximization (EM) approach that considers the missing valu…
Relationship between agents can be conveniently represented by graphs. When these relationships have different modalities, they are better modelled by multilayer graphs where each layer is associated with one modality. Such graphs arise naturally in many contexts including biological and social networks. Clustering is a fundamental problem in network analysis where the goal is to regroup nodes with similar connectivity profiles. In the past decade, various clustering methods have been extended from the unilayer setting to multilayer graphs in order to incorporate the information provided by ea…
Nanyang Technological University（南洋理工大学）；DSO National Laboratories（国防科技组织国家实验室）；Macquarie University（麦考瑞大学）
Clustering evaluation measures are frequently used to evaluate the performance of algorithms. However, most measures are not properly normalized and ignore some information in the inherent structure of clusterings. We model the relation between two clusterings as a bipartite graph and propose a general component-based decomposition formula based on the components of the graph. Most existing measures are examples of this formula. In order to satisfy consistency in the component, we further propose a split-merge framework for comparing clusterings of different data sets. Our framework gives meas…
Boston University（波士顿大学）；University of Wisconsin - Madison（威斯康星大学麦迪逊分校）
This paper considers the problem of completing a matrix with many missing entries under the assumption that the columns of the matrix belong to a union of multiple low-rank subspaces. This generalizes the standard low-rank matrix completion problem to situations in which the matrix rank can be quite high or even full rank. Since the columns belong to a union of subspaces, this problem may also be viewed as a missing-data version of the subspace clustering problem. Let X be an n x N matrix whose (complete) columns lie in a union of at most k subspaces, each of rank <= r < n, and assume N >> kn.…
Centre Tecnològic Telecomunicacions Catalunya（加泰罗尼亚电信技术中心）；University of Padova（帕多瓦大学）；Aalborg University（奥尔堡大学）；Software Radio Systems（软件无线电系统公司）
Hierarchical Rate Splitting (HRS) schemes proposed in recent years have shown to provide significant improvements in exploiting spatial diversity in wireless networks and provide high throughput for all users while minimising interference among them. Hence, one of the major challenges for such HRS schemes is the necessity to know the optimal clustering of these users based only on their Channel State Information (CSI). This clustering problem is known to be NP hard and, to deal with the unmanageable complexity of finding an optimal solution, in this work a scalable and much lighter clustering …
Stable Trajectory Clustering: An Efficient Split and Merge Algorithm
Department of Computer Science and Information Technology, Institute for Advanced Studies in Basic Sciences (IASBS)（计算机科学与信息技术系，基础科学研究所（IASBS））；Center for Advanced Systems Understanding (CASUS), Helmholtz-Zentrum Dresden-Rossendorf（先进系统理解中心（CASUS），德累斯顿-罗斯托克研究中心）；Helmholtz Centre for Environmental Research – UFZ（环境研究赫尔姆霍兹中心 – UFZ）；University of Maryland（马里兰大学）
University of Illinois at Chicago（伊利诺伊大学芝加哥分校）；Shenzhen University（深圳大学）
In the era of big data, it is common to have data with multiple modalities or coming from multiple sources, known as "multi-view data". Multi-view clustering provides a natural way to generate clusters from such data. Since different views share some consistency and complementary information, previous works on multi-view clustering mainly focus on how to combine various numbers of views to improve clustering performance. However, in reality, each view may be incomplete, i.e., instances missing in the view. Furthermore, the size of data could be extremely huge. It is unrealistic to apply multi-…
Hierarchical Mutual Information Analysis: Towards Multi-view Clustering in The Wild
OPPO Research Institute（OPPO研究院）；Xinchuang Haihe Laboratory（信创海河实验室）；Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）；InnoPeak Technology, Inc（InnoPeak Technology公司）；Stony Brook University（石溪大学）
Interpretable clustering via optimal multi-way decision trees
Graduate School of Science and Technology, University of Tsukuba（科学技术研究生院，东京大学）；Institute of Systems and Information Engineering, University of Tsukuba（系统与信息工程研究所，东京大学）
Modern inference and learning often hinge on identifying low-dimensional structures that approximate large scale data. Subspace clustering achieves this through a union of linear subspaces. However, in contemporary applications data is increasingly often incomplete, rendering standard (full-data) methods inapplicable. On the other hand, existing incomplete-data methods present major drawbacks, like lifting an already high-dimensional problem, or requiring a super polynomial number of samples. Motivated by this, we introduce a new subspace clustering algorithm inspired by fusion penalties. The …
A Unified Framework for Variable Selection in Model-Based Clustering with Missing Not at Random
Faculty of Mathematics and Computer Science, University of Science, Ho Chi Minh City, Vietnam（越南科技大学数学与计算机科学学院）；Vietnam National University Ho Chi Minh City, Vietnam（越南胡志明市国家大学）；School of Information and Communications Technology, Hanoi University of Science and Technology, Ha Noi, Vietnam（河内科学技术大学信息与通信技术学院）；ARC Centre of Excellence for the Mathematical Analysis of Cellular Systems（细胞系统数学分析卓越中心）；School of Mathematical Sciences, Queensland University of Technology, Brisbane City, Australia（昆士兰科技大学数学科学学院）
Divide-Then-Rule: A Cluster-Driven Hierarchical Interpolator for Attribute-Missing Graphs
College of Computer Science and Technology（计算机科学与技术学院）；National University of Defense Technology（国防科技大学）；Xiangjiang Laboratory（湘江实验室）；Hainan University（海南大学）；Changsha University（长沙大学）；Qilu University of Technology（齐鲁工业大学）；North University of China（北方工业大学）
School of Artificial Intelligence, OPtics and ElectroNics (iOPEN)（人工智能学院（iOPEN））；Northwestern Polytechnical University（西北工业大学）；Institute of Artificial Intelligence (TeleAI)（人工智能研究所（TeleAI））
University of California, Irvine（加州大学欧文分校）；University of Algarve（阿尔加维大学）
Motivation: With the development of droplet based systems, massive single cell transcriptome data has become available, which enables analysis of cellular and molecular processes at single cell resolution and is instrumental to understanding many biological processes. While state-of-the-art clustering methods have been applied to the data, they face challenges in the following aspects: (1) the clustering quality still needs to be improved; (2) most models need prior knowledge on number of clusters, which is not always available; (3) there is a demand for faster computational speed. Results: We…
Simon Fraser University（西蒙弗雷泽大学）；Ulsan National Institute of Science and Technology（蔚山科学技术院）；University of British Columbia（不列颠哥伦比亚大学）；Disney Research（迪士尼研究院）
We present a hierarchical maximum-margin clustering method for unsupervised data analysis. Our method extends beyond flat maximum-margin clustering, and performs clustering recursively in a top-down manner. We propose an effective greedy splitting criteria for selecting which cluster to split next, and employ regularizers that enforce feature sharing/competition for capturing data semantics. Experimental results obtained on four standard datasets show that our method outperforms flat and hierarchical clustering baselines, while forming clean and semantically meaningful cluster hierarchies.
SPORT: Structure-Aware Prototype Disentanglement for Incomplete Multi-View Clustering
College of Computer and Cyberspace Security, Hebei Normal University（河北师范大学计算机与网络空间安全学院）；Department of Statistics and Data Science, Southern University of Science and Technology（南方科技大学统计与数据科学系）；Key Laboratory of Tibetan Information Processing, Ministry of Education, Qinghai Normal University（青海师范大学藏文信息处理教育部重点实验室）；School of Computer Science and Technology, Beijing Jiaotong University（北京交通大学计算机科学与技术学院）；State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所多模态人工智能系统国家重点实验室）
A new type of federated clustering: A non-model-sharing approach
Center for Artificial Intelligence Research（人工智能研究中心）；Tsukuba Institute for Advanced Research（茨口高级研究机构）；University of Tsukuba（茨口大学）；Graduate School of Science and Technology（科学与技术研究生院）；Institute of Systems and Information Engineering（系统与信息工程研究所）
Eli Lilly and Company（礼来公司）；Iowa State University（爱荷华州立大学）
The $k$-means algorithm is arguably the most popular nonparametric clustering method but cannot generally be applied to datasets with incomplete records. The usual practice then is to either impute missing values under an assumed missing-completely-at-random mechanism or to ignore the incomplete records, and apply the algorithm on the resulting dataset. We develop an efficient version of the $k$-means algorithm that allows for clustering in the presence of incomplete records. Our extension is called $k_m$-means and reduces to the $k$-means algorithm when all records are complete. We also provi…
Southwest Jiaotong University（西南交通大学）；University of Illinois at Chicago（伊利诺伊大学芝加哥分校）；Dalian University of Technology（大连理工大学）
Beyond existing multi-view clustering, this paper studies a more realistic clustering scenario, referred to as incomplete multi-view clustering, where a number of data instances are missing in certain views. To tackle this problem, we explore spectral perturbation theory. In this work, we show a strong link between perturbation risk bounds and incomplete multi-view clustering. That is, as the similarity matrix fed into spectral clustering is a quantity bounded in magnitude O(1), we transfer the missing problem from data to similarity and tailor a matrix completion method for incomplete similar…
ANIMC: A Soft Framework for Auto-weighted Noisy and Incomplete Multi-view Clustering
Hubei Engineering Research Center on Big Data Security, School of Cyber Science and Engineering, Huazhong University of Science and Technology（大数据安全湖北工程研究中心，信息科学与工程学院，华中科技大学）；School of Computer Science and Technology, Huazhong University of Science and Technology（计算机科学与技术学院，华中科技大学）；Key Laboratory of Information Storage System Ministry of Education of China, Huazhong University of Science and Technology（信息存储系统教育部重点实验室，华中科技大学）；Department of Electrical and Computer Engineering, University of Florida（电气与计算机工程系，佛罗里达大学）
University of Salerno（萨勒诺大学）；IMT School for Advanced Studies（IMT高等研究院）
World Input-Output (I/O) matrices provide the networks of within- and cross-country economic relations. In the context of I/O analysis, the methodology adopted by national statistical offices in data collection raises the issue of obtaining reliable data in a timely fashion and it makes the reconstruction of (part of) the I/O matrices of particular interest. In this work, we propose a method combining hierarchical clustering and Matrix Completion (MC) with a LASSO-like nuclear norm penalty, to impute missing entries of a partially unknown I/O matrix. Through simulations based on synthetic matr…
University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；MIT-IBM Watson AI Lab（MIT-IBM沃森人工智能实验室）
In the era of big data, we are often facing the challenge of data heterogeneity and the lack of label information simultaneously. In the financial domain (e.g., fraud detection), the heterogeneous data may include not only numerical data (e.g., total debt and yearly income), but also text and images (e.g., financial statement and invoice images). At the same time, the label information (e.g., fraud transactions) may be missing for building predictive models. To address these challenges, many state-of-the-art multi-view clustering methods have been proposed and achieved outstanding performance.…
UiT The Arctic University of Norway（挪威北极大学）；Universidad Rey Juan Carlos（胡安卡洛斯国王大学）
Similarity-based approaches represent a promising direction for time series analysis. However, many such methods rely on parameter tuning, and some have shortcomings if the time series are multivariate (MTS), due to dependencies between attributes, or the time series contain missing data. In this paper, we address these challenges within the powerful context of kernel methods by proposing the robust \emph{time series cluster kernel} (TCK). The approach taken leverages the missing data handling properties of Gaussian mixture models (GMM) augmented with informative prior distributions. An ensemb…
University of California, Riverside（加州大学河滨分校）；University of New Mexico（新墨西哥大学）
The discovery of time series motifs has emerged as one of the most useful primitives in time series data mining. Researchers have shown its utility for exploratory data mining, summarization, visualization, segmentation, classification, clustering, and rule discovery. Although there has been more than a decade of extensive research, there is still no technique to allow the discovery of time series motifs in the presence of missing data, despite the well-documented ubiquity of missing data in scientific, industrial, and medical datasets. In this work, we introduce a technique for motif discover…
