Adiabatic quantum computers are a promising platform for approximately solving challenging optimization problems. We present a quantum approach to solving the balanced $k$-means clustering training problem on the D-Wave 2000Q adiabatic quantum computer. Existing classical approaches scale poorly for large datasets and only guarantee a locally optimal solution. We show that our quantum approach better targets the global solution of the training problem, while achieving better theoretic scalability on large datasets. We test our quantum approach on a number of small problems, and observe clustering performance similar to the best classical algorithms.
The University of Queensland（昆士兰大学）；University of Texas（德克萨斯大学）；Carnegie Mellon University（卡内基梅隆大学）
Clustering is an effective technique in data mining to generate groups that are the matter of interest. Among various clustering approaches, the family of k-means algorithms and min-cut algorithms gain most popularity due to their simplicity and efficacy. The classical k-means algorithm partitions a number of data points into several subsets by iteratively updating the clustering centers and the associated data points. By contrast, a weighted undirected graph is constructed in min-cut algorithms which partition the vertices of the graph into two sets. However, existing clustering algorithms te…
Michigan State University（密歇根州立大学）；University of Science and Technology of China（中国科学技术大学）
The problem of constrained clustering has attracted significant attention in the past decades. In this paper, we study the balanced $k$-center, $k$-median, and $k$-means clustering problems where the size of each cluster is constrained by the given lower and upper bounds. The problems are motivated by the applications in processing large-scale data in high dimension. Existing methods often need to compute complicated matchings (or min cost flows) to satisfy the balance constraint, and thus suffer from high complexities especially in high dimension. We develop an effective framework for the thr…
Balanced order batching problem (BOBP) arises from the process of warehouse picking in Cainiao, the largest logistics platform in China. Batching orders together in the picking process to form a single picking route, reduces travel distance. The reason for its importance is that order picking is a labor intensive process and, by using good batching methods, substantial savings can be obtained. The BOBP is a NP-hard combinational optimization problem and designing a good problem-specific heuristic under the quasi-real-time system response requirement is non-trivial. In this paper, rather than d…
City University of Macau（澳门城市大学）；National University of Defense Technology（国防科技大学）；China University of Geosciences（中国地质大学）；Zhejiang Normal University（浙江师范大学）；Research Institute of Ningbo Cixing Co. Ltd（宁波慈星股份有限公司研究院）；Qilu University of Technology (Shandong Academy of Sciences)（齐鲁工业大学（山东省科学院））；Shandong Computer Science Center (National Supercomputer Center in Jinan)（山东省计算中心（国家超级计算济南中心））；Shandong Fundamental Research Center for Computer Science（山东省计算机科学基础研究中心）
Spectral Clustering as a relaxation of the normalized/ratio cut has become one of the standard graph-based clustering methods. Existing methods for the computation of multiple clusters, corresponding to a balanced $k$-cut of the graph, are either based on greedy techniques or heuristics which have weak connection to the original motivation of minimizing the normalized cut. In this paper we propose a new tight continuous relaxation for any balanced $k$-cut problem and show that a related recently proposed relaxation is in most cases loose leading to poor performance in practice. For the optimiz…
Technische Universität München（慕尼黑工业大学）；University of Colorado Denver（科罗拉多大学丹佛分校）；Universität der Bundeswehr München（慕尼黑联邦国防军大学）
The classical $k$-means algorithm for partitioning $n$ points in $\mathbb{R}^d$ into $k$ clusters is one of the most popular and widely spread clustering methods. The need to respect prescribed lower bounds on the cluster sizes has been observed in many scientific and business applications. In this paper, we present and analyze a generalization of $k$-means that is capable of handling weighted point sets and prescribed lower and upper bounds on the cluster sizes. We call it weight-balanced $k$-means. The key difference to existing models lies in the ability to handle the combination of weighte…
Unsupervised domain adaptation seeks to learn an invariant and discriminative representation for an unlabeled target domain by leveraging the information of a labeled source dataset. We propose to improve the discriminative ability of the target domain representation by simultaneously learning tightly clustered target representations while encouraging that each cluster is assigned to a unique and different class from the source. This strategy alleviates the effects of negative transfer when combined with adversarial domain matching between source and target representations. Our approach is rob…
Towards Cohesion-Fairness Harmony: Contrastive Regularization in Individual Fair Graph Clustering
L3S Research Centre, Leibniz University Hannover（莱布尼茨汉诺威大学L3S研究中心）；University of Kurdistan（库尔德斯坦大学）；RI CODE, University of the Bundeswehr Munich（慕尼黑联邦国防军大学RI CODE）
Leibniz University Hannover（汉诺威莱布尼茨大学）；Free University of Berlin（柏林自由大学）
Traditionally, clustering algorithms focus on partitioning the data into groups of similar instances. The similarity objective, however, is not sufficient in applications where a fair-representation of the groups in terms of protected attributes like gender or race, is required for each cluster. Moreover, in many applications, to make the clusters useful for the end-user, a balanced cardinality among the clusters is required. Our motivation comes from the education domain where studies indicate that students might learn better in diverse student groups and of course groups of similar cardinali…
The University of Texas at Austin（得克萨斯大学奥斯汀分校）；Google Research（谷歌研究院）
We propose a novel type of balanced clustering algorithm to approximate attention. Attention complexity is reduced from $O(N^2)$ to $O(N \log N)$, where $N$ is the sequence length. Our algorithm, SMYRF, uses Locality Sensitive Hashing (LSH) in a novel way by defining new Asymmetric transformations and an adaptive scheme that produces balanced clusters. The biggest advantage of SMYRF is that it can be used as a drop-in replacement for dense attention layers without any retraining. On the contrary, prior fast attention methods impose constraints (e.g. queries and keys share the same vector repre…
We consider the unsupervised learning problem of assigning labels to unlabeled data. A naive approach is to use clustering methods, but this works well only when data is properly clustered and each cluster corresponds to an underlying class. In this paper, we first show that this unsupervised labeling problem in balanced binary cases can be solved if two unlabeled datasets having different class balances are available. More specifically, estimation of the sign of the difference between probability densities of two unlabeled datasets gives the solution. We then introduce a new method to directl…
College of Computer Studies De La Salle University（德拉萨大学计算机研究学院）
Deep clustering algorithms combine representation learning and clustering by jointly optimizing a clustering loss and a non-clustering loss. In such methods, a deep neural network is used for representation learning together with a clustering network. Instead of following this framework to improve clustering performance, we propose a simpler approach of optimizing the entanglement of the learned latent code representation of an autoencoder. We define entanglement as how close pairs of points from the same class or structure are, relative to pairs of points from different classes or structures.…
University of Toronto（多伦多大学）；Fujitsu Laboratories of America, Inc.（美国富士通研究所）
The emergence of specialized optimization hardware such as CMOS annealers and adiabatic quantum computers carries the promise of solving hard combinatorial optimization problems more efficiently in hardware. Recent work has focused on formulating different combinatorial optimization problems as Ising models, the core mathematical abstraction used by a large number of these hardware platforms, and evaluating the performance of these models when solved on specialized hardware. An interesting area of application is data mining, where combinatorial optimization problems underlie many core tasks. I…
UT-Battelle, LLC（UT-Battelle有限责任公司）；U.S. Department of Energy（美国能源部）；Oak Ridge Leadership Computing Facility（橡树岭领导计算设施）；DOE Office of Science（美国能源部科学办公室）；Oak Ridge National Laboratory（橡树岭国家实验室）；Auburn University（奥本大学）；Washington University in St. Louis（圣路易斯华盛顿大学）
Training machine learning models on classical computers is usually a time and compute intensive process. With Moore's law coming to an end and ever increasing demand for large-scale data analysis using machine learning, we must leverage non-conventional computing paradigms like quantum computing to train machine learning models efficiently. Adiabatic quantum computers like the D-Wave 2000Q can approximately solve NP-hard optimization problems, such as the quadratic unconstrained binary optimization (QUBO), faster than classical computers. Since many machine learning problems are also NP-hard, …
Assignment methods are at the heart of many algorithms for unsupervised learning and clustering - in particular, the well-known K-means and Expectation-Maximization (EM) algorithms. In this work, we study several different methods of assignment, including the "hard" assignments used by K-means and the ?soft' assignments used by EM. While it is known that K-means minimizes the distortion on the data and EM maximizes the likelihood, little is known about the systematic differences of behavior between the two algorithms. Here we shed light on these differences via an information-theoretic analysi…
Università della Svizzera italiana（提契诺大学）；Friedrich-Alexander-Universität Erlangen-Nürnberg（埃尔朗根-纽伦堡大学）
Nonlinear reformulations of the spectral clustering method have gained a lot of recent attention due to their increased numerical benefits and their solid mathematical background. We present a novel direct multiway spectral clustering algorithm in the $p$-norm, for $p \in (1, 2]$. The problem of computing multiple eigenvectors of the graph $p$-Laplacian, a nonlinear generalization of the standard graph Laplacian, is recasted as an unconstrained minimization problem on a Grassmann manifold. The value of $p$ is reduced in a pseudocontinuous manner, promoting sparser solution vectors that corresp…
Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究院）；Tianjin University of Technology（天津理工大学）；National and Kapodistrian University of Athens（雅典国立卡波季斯特里安大学）；The University of Hong Kong（香港大学）
Clustering is one of the most fundamental tools in the artificial intelligence area, particularly in the pattern recognition and learning theory. In this paper, we propose a simple, but novel approach for variance-based k-clustering tasks, included in which is the widely known k-means clustering. The proposed approach picks a sampling subset from the given dataset and makes decisions based on the data information in the subset only. With certain assumptions, the resulting clustering is provably good to estimate the optimum of the variance-based objective with high probability. Extensive experi…
In this work, we study the $k$-median and $k$-means clustering problems when the data is distributed across many servers and can contain outliers. While there has been a lot of work on these problems for worst-case instances, we focus on gaining a finer understanding through the lens of beyond worst-case analysis. Our main motivation is the following: for many applications such as clustering proteins by function or clustering communities in a social network, there is some unknown target clustering, and the hope is that running a $k$-median or $k$-means algorithm will produce clusterings which …
School of Microelectronics and Communication Engineering, Chongqing University（重庆大学微电子与通信工程学院）
The challenge of imbalanced learning lies not only in class imbalance problem, but also in the class overlapping problem which is complex. However, most of the existing algorithms mainly focus on the former. The limitation prevents the existing methods from breaking through. To address this limitation, this paper proposes an ensemble learning algorithm based on dual clustering and stage-wise hybrid sampling (DCSHS). The DCSHS has three parts. Firstly, we design a projection clustering combination framework (PCC) guided by Davies-Bouldin clustering effectiveness index (DBI), which is used to ob…
Constant-Factor Approximations for Doubly Constrained Fair k-Center, k-Median and k-Means
Department of Informatics, University of Cologne, Germany（科隆大学信息学院，德国）；Heinrich Heine University Düsseldorf, Faculty of Mathematics and Natural Sciences, Germany（杜伊斯堡-艾森大学数学与自然科学学院，德国）；Department of Informatics, University of Bonn, Germany（波恩大学信息学院，德国）
