Low-Rank Representation (LRR) highly suffers from discarding the locality information of data points in subspace clustering, as it may not incorporate the data structure nonlinearity and the non-uniform distribution of observations over the ambient space. Thus, the information of the observational density is lost by the state-of-art LRR models, as they take a constant number of adjacent neighbors into account. This, as a result, degrades the subspace clustering accuracy in such situations. To cope with deficiency, in this paper, we propose to consider a hypergraph model to facilitate having a variable number of adjacent nodes and incorporating the locality information of the data. The sparsity of the number of subspaces is also taken into account. To do so, an optimization problem is defined based on a set of regularization terms and is solved by developing a tensor Laplacian-based algorithm. Extensive experiments on artificial and real datasets demonstrate the higher accuracy and precision of the proposed method in subspace clustering compared to the state-of-the-art methods. The outperformance of this method is more revealed in presence of inherent structure of the data such as nonlinearity, geometrical overlapping, and outliers.
China University of Mining and Technology（中国矿业大学）；Nankai University（南开大学）
Spectral clustering methods are widely used for detecting clusters in networks for community detection, while a small change on the graph Laplacian matrix could bring a dramatic improvement. In this paper, we propose a dual regularized graph Laplacian matrix and then employ it to three classical spectral clustering approaches under the degree-corrected stochastic block model. If the number of communities is known as $K$, we consider more than $K$ leading eigenvectors and weight them by their corresponding eigenvalues in the spectral clustering procedure to improve the performance. Three improv…
We provide a theoretical analysis of the representation learning problem aimed at learning the latent variables (design matrix) $Θ$ of observations $Y$ with the knowledge of the coefficient matrix $X$. The design matrix is learned under the assumption that the latent variables $Θ$ are smooth with respect to a (known) topological structure $\mathcal{G}$. To learn such latent variables, we study a graph Laplacian regularized estimator, which is the penalized least squares estimator with penalty term proportional to a Laplacian quadratic form. This type of estimators has recently received conside…
Inria（法国国家信息与自动化研究所）；Trapeznikov Institute of Control Sciences of the Russian Academy of Sciences（俄罗斯科学院特拉佩兹尼科夫控制科学研究所）；St. Petersburg State University（圣彼得堡国立大学）
We study a semi-supervised learning method based on the similarity graph and RegularizedLaplacian. We give convenient optimization formulation of the Regularized Laplacian method and establishits various properties. In particular, we show that the kernel of the methodcan be interpreted in terms of discrete and continuous time random walks and possesses several importantproperties of proximity measures. Both optimization and linear algebra methods can be used for efficientcomputation of the classification functions. We demonstrate on numerical examples that theRegularized Laplacian method is co…
We consider the problem of jointly estimating multiple related zero-mean Gaussian distributions from data. We propose to jointly estimate these covariance matrices using Laplacian regularized stratified model fitting, which includes loss and regularization terms for each covariance matrix, and also a term that encourages the different covariances matrices to be close. This method `borrows strength' from the neighboring covariances, to improve its estimate. With well chosen hyper-parameters, such models can perform very well, especially in the low data regime. We propose a distributed method th…
We propose a transductive Laplacian-regularized inference for few-shot tasks. Given any feature embedding learned from the base classes, we minimize a quadratic binary-assignment function containing two terms: (1) a unary term assigning query samples to the nearest class prototype, and (2) a pairwise Laplacian term encouraging nearby query samples to have consistent label assignments. Our transductive inference does not re-train the base model, and can be viewed as a graph clustering of the query set, subject to supervision constraints from the support set. We derive a computationally efficien…
China University of Mining and Technology（中国矿业大学）；Nankai University（南开大学）
Community detection in network analysis is an attractive research area recently. Here, under the degree-corrected mixed membership (DCMM) model, we propose an efficient approach called mixed regularized spectral clustering (Mixed-RSC for short) based on the regularized Laplacian matrix. Mixed-RSC is designed based on an ideal cone structure of the variant for the eigen-decomposition of the population regularized Laplacian matrix. We show that the algorithm is asymptotically consistent under mild conditions by providing error bounds for the inferred membership vector of each node. As a byproduc…
University College London（伦敦大学学院）；University of Oxford（牛津大学）
We consider a stochastic linear bandit problem with multiple users, where the relationship between users is captured by an underlying graph and user preferences are represented as smooth signals on the graph. We introduce a novel bandit algorithm where the smoothness prior is imposed via the random-walk graph Laplacian, which leads to a single-user cumulative regret scaling as $\tilde{\mathcal{O}}(Ψd \sqrt{T})$ with time horizon $T$, feature dimensionality $d$, and the scalar parameter $Ψ\in (0,1)$ that depends on the graph connectivity. This is an improvement over $\tilde{\mathcal{O}}(d \sqrt…
Stratified models are models that depend in an arbitrary way on a set of selected categorical features, and depend linearly on the other features. In a basic and traditional formulation a separate model is fit for each value of the categorical feature, using only the data that has the specific categorical value. To this formulation we add Laplacian regularization, which encourages the model parameters for neighboring categorical values to be similar. Laplacian regularization allows us to specify one or more weighted graphs on the stratification feature values. For example, stratifying over the…
the Hong Kong Polytechnic University（香港理工大学）；Harbin Institute of Technology（哈尔滨工业大学）
Multi-view subspace learning (MSL) aims to find a low-dimensional subspace of the data obtained from multiple views. Different from single view case, MSL should take both common and specific knowledge among different views into consideration. To enhance the robustness of model, the complexity, non-consistency and similarity of noise in multi-view data should be fully taken into consideration. Most current MSL methods only assume a simple Gaussian or Laplacian distribution for the noise while neglect the complex noise configurations in each view and noise correlations among different views of p…
Shell Information Technology Limited（Shell信息科技有限公司）；University College London（伦敦大学学院）；Department of Computer Science（计算机科学系）；Systemic Risk Centre（系统性风险中心）；London School of Economics and Political Sciences（伦敦政治经济学院）；Shell Information Technology International B.V.（Shell信息科技国际有限公司）
Alternating minimization algorithms for graph regularized tensor completion
Huawei European Research Institute（华为欧洲研究院）；LISN, INRIA, Université Paris-Saclay（法国国家信息与自动化研究所，巴黎萨克雷大学）；LSEC, Academy of Mathematics and Systems Science, Chinese Academy of Sciences（中国科学院数学与系统科学研究院科学与工程计算国家重点实验室）；Department of INMA, Université catholique de Louvain（鲁汶天主教大学）
University of Edinburgh（爱丁堡大学）；Edinburgh Cancer Centre, Western General Hospital（西部综合医院爱丁堡癌症中心）
Computer-aided breast cancer diagnosis in mammography is limited by inadequate data and the similarity between benign and cancerous masses. To address this, we propose a signed graph regularized deep neural network with adversarial augmentation, named \textsc{DiagNet}. Firstly, we use adversarial learning to generate positive and negative mass-contained mammograms for each mass class. After that, a signed similarity graph is built upon the expanded data to further highlight the discrimination. Finally, a deep convolutional neural network is trained by jointly optimizing the signed graph regula…
University of Oxford（牛津大学）；The Alan Turing Institute（阿兰·图灵研究所）；New York University（纽约大学）；Inria（法国国家信息与自动化研究所）；Univ. Lille（里尔大学）；CNRS（法国国家科学研究中心）
We study the problem of $k$-way clustering in signed graphs. Considerable attention in recent years has been devoted to analyzing and modeling signed graphs, where the affinity measure between nodes takes either positive or negative values. Recently, Cucuringu et al. [CDGT 2019] proposed a spectral method, namely SPONGE (Signed Positive over Negative Generalized Eigenproblem), which casts the clustering task as a generalized eigenvalue problem optimizing a suitably defined objective function. This approach is motivated by social balance theory, where the clustering task aims to decompose a giv…
Analysis of high dimensional noisy data is of essence across a variety of research fields. Feature selection techniques are designed to find the relevant feature subset that can facilitate classification or pattern detection. Traditional (supervised) feature selection methods utilize label information to guide the identification of relevant feature subsets. In this paper, however, we consider the unsupervised feature selection problem. Without the label information, it is particularly difficult to identify a small set of relevant features due to the noisy nature of real-world data which corrup…
Fudan University（复旦大学）；School of Information Science and Technology（信息科学与工程学院）
Capturing complex high-order interactions among data is an important task in many scenarios. A common way to model high-order interactions is to use hypergraphs whose topology can be mathematically represented by tensors. Existing methods use a fixed-order tensor to describe the topology of the whole hypergraph, which ignores the divergence of different-order interactions. In this work, we take this divergence into consideration, and propose a multi-order hypergraph Laplacian and the corresponding total variation. Taking this total variation as a regularization term, we can utilize the topolog…
Multi-relational learning has received lots of attention from researchers in various research communities. Most existing methods either suffer from superlinear per-iteration cost, or are sensitive to the given ranks. To address both issues, we propose a scalable core tensor trace norm Regularized Orthogonal Iteration Decomposition (ROID) method for full or incomplete tensor analytics, which can be generalized as a graph Laplacian regularized version by using auxiliary information or a sparse higher-order orthogonal iteration (SHOOI) version. We first induce the equivalence relation of the Scha…
Worcester Polytechnic Institute（伍斯特理工学院）；University of Bologna（博洛尼亚大学）
In this paper we use neural networks to learn governing equations from data. Specifically we reconstruct the right-hand side of a system of ODEs $\dot{x}(t) = f(t, x(t))$ directly from observed uniformly time-sampled data using a neural network. In contrast with other neural network based approaches to this problem, we add a Lipschitz regularization term to our loss function. In the synthetic examples we observed empirically that this regularization results in a smoother approximating function and better generalization properties when compared with non-regularized models, both on trajectory an…
The Fiedler vector of a connected graph is the eigenvector associated with the algebraic connectivity of the graph Laplacian and it provides substantial information to learn the latent structure of a graph. In real-world applications, however, the data may be subject to heavy-tailed noise and outliers which results in deteriorations in the structure of the Fiedler vector estimate. We design a Robust Regularized Locality Preserving Indexing (RRLPI) method for Fiedler vector estimation that aims to approximate the nonlinear manifold structure of the Laplace Beltrami operator while minimizing the…
Multiview canonical correlation analysis (MCCA) seeks latent low-dimensional representations encountered with multiview data of shared entities (a.k.a. common sources). However, existing MCCA approaches do not exploit the geometry of the common sources, which may be available \emph{a priori}, or can be constructed using certain domain knowledge. This prior information about the common sources can be encoded by a graph, and be invoked as a regularizer to enrich the maximum variance MCCA framework. In this context, the present paper's novel graph-regularized (G) MCCA approach minimizes the dista…
