In machine learning, observation features are measured in a metric space to obtain their distance function for optimization. Given similar features that are statistically sufficient as a population, a statistical distance between two probability distributions can be calculated for more precise learning. Provided the observed features are multi-valued, the statistical distance function is still efficient. However, due to its scalar output, it cannot be applied to represent detailed distances between feature elements. To resolve this problem, this paper extends the traditional statistical distance to a matrix form, called a statistical distance matrix. In experiments, the proposed approach performs well in object recognition tasks and clearly and intuitively represents the dissimilarities between cat and dog images in the CIFAR dataset, even when directly calculated using the image pixels. By using the hierarchical clustering of the statistical distance matrix, the image pixels can be separated into several clusters that are geometrically arranged around a center like a Mandala pattern. The statistical distance matrix with clustering, called the Information Mandala, is beyond ordinary saliency maps and can help to understand the basic principles of the convolution neural network.
Statistical Inference for Clustering-based Anomaly Detection
University of Information Technology, Ho Chi Minh City, Vietnam（越南胡志明市信息技术大学）；Vietnam National University, Ho Chi Minh City, Vietnam（越南国家大学）；RIKEN, Japan（日本理化学研究所）
University of Tübingen（图宾根大学）；Max Planck Institute for Intelligent Systems（马克斯·普朗克智能系统研究所）；Technical University of Munich（慕尼黑工业大学）
Despite the ubiquity of kernel-based clustering, surprisingly few statistical guarantees exist beyond settings that consider strong structural assumptions on the data generation process. In this work, we take a step towards bridging this gap by studying the statistical performance of kernel-based clustering algorithms under non-parametric mixture models. We provide necessary and sufficient separability conditions under which these algorithms can consistently recover the underlying true clustering. Our analysis provides guarantees for kernel clustering approaches without structural assumptions …
High-order clustering aims to identify heterogeneous substructures in multiway datasets that arise commonly in neuroimaging, genomics, social network studies, etc. The non-convex and discontinuous nature of this problem pose significant challenges in both statistics and computation. In this paper, we propose a tensor block model and the computationally efficient methods, \emph{high-order Lloyd algorithm} (HLloyd), and high-order spectral clustering (HSC), for high-order clustering. The convergence guarantees and statistical optimality are established for the proposed procedure under a mild sub…
Aalto University（阿尔托大学）；Sharif University of Technology（谢里夫理工大学）
We study the statistical and computational properties of a network Lasso method for local graph clustering. The clusters delivered by nLasso can be characterized elegantly via network flows between cluster boundary and seed nodes. While spectral clustering methods are guided by a minimization of the graph Laplacian quadratic form, nLasso minimizes the total variation of cluster indicator signals. As demonstrated theoretically and numerically, nLasso methods can handle very sparse clusters (chain-like) which are difficult for spectral clustering. We also verify that a primal-dual method for non…
Massachusetts Institute of Technology（麻省理工学院）；Courant Institute of Mathematical Sciences, New York University（纽约大学库朗数学科学研究所）；Simons Institute for the Theory of Computing, UC Berkeley（加州大学伯克利分校西蒙斯计算理论研究所）；Center for Data Science, New York University（纽约大学数据科学中心）；Flatiron Institute（熨斗研究所）
Clustering is a fundamental primitive in unsupervised learning which gives rise to a rich class of computationally-challenging inference tasks. In this work, we focus on the canonical task of clustering d-dimensional Gaussian mixtures with unknown (and possibly degenerate) covariance. Recent works (Ghosh et al. '20; Mao, Wein '21; Davis, Diaz, Wang '21) have established lower bounds against the class of low-degree polynomial methods and the sum-of-squares (SoS) hierarchy for recovering certain hidden structures planted in Gaussian clustering instances. Prior work on many similar inference task…
Model-Based Clustering and Classification of Functional Data
Normandie Univ, UNICAEN, UMR CNRS LMNO, Department of Mathematics and Computer Science（诺曼底大学、卡昂大学、法国国家科学研究中心LMNO联合研究单位、数学与计算机科学系）；Department of Mathematics and Statistics, La Trobe University（拉筹伯大学数学与统计系）；La Trobe University（拉筹伯大学）
Topology of Currencies: Persistent Homology for FX Co-movements: A Comparative Clustering Study
Maastricht University, School of Business and Economics（马斯特里赫特大学商学院）；Department of Data Analytics and Digitalisation（数据分析与数字化部门）；Department of Data Analytics（数据分析部门）
Courant Institute of Mathematical Sciences, NYU（纽约大学柯朗数学科学研究所）
We study statistical and computational limits of clustering when the means of the centres are sparse and their dimension is possibly much larger than the sample size. Our theoretical analysis focuses on the model $X_i = z_i θ+ \varepsilon_i, ~z_i \in \{-1,1\}, ~\varepsilon_i \thicksim \mathcal{N}(0,I)$, which has two clusters with centres $θ$ and $-θ$. We provide a finite sample analysis of a new sparse clustering algorithm based on sparse PCA and show that it achieves the minimax optimal misclustering rate in the regime $\|θ\| \rightarrow \infty$. Our results require the sparsity to grow slow…
