arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2412.04082cs.LG
收藏

可学习相似性与差异性引导的对称非负矩阵分解

Learnable Similarity and Dissimilarity Guided Symmetric Non-Negative Matrix Factorization

School of Computer Science and Engineering, Southeast University(东南大学计算机科学与工程学院)

Wenlong Lyu, Yuheng Jia

AI总结:

针对SymNMF中k-NN构建相似性矩阵存在的误导问题,提出带可学习权重的加权k-NN图构建方法,引入对偶差异性矩阵与新正交正则化,设计交替优化算法并理论证明收敛性,在8个数据集上优于9种先进方法。

摘要:

对称非负矩阵分解(SymNMF)是一种强大的聚类工具,通常使用k近邻(k-NN)方法构建相似性矩阵。然而,k-NN可能会误导聚类,因为邻居可能属于不同的簇,且其可靠性通常随k增大而降低。本文将相似性矩阵构建为带可学习权重的加权k-NN图,该权重反映每个第k个近邻的可靠性。此方法将相似性矩阵学习的搜索空间从现有方法的O(n²)维减少到n-1维,其中n表示样本数量。此外,为获得具有判别性的相似性矩阵,我们引入与相似性矩阵具有对偶结构的差异性矩阵,并提出一种新形式的正交正则化,同时讨论其几何解释和数值稳定性。我们设计了一种高效的交替优化算法来求解所提出的模型,并从理论上证明变量收敛到满足KKT条件的平稳点。通过在8个数据集上与9种最先进的聚类方法进行比较,验证了所提出模型的优势。代码可在https://github.com/lwl-learning/LSDGSymNMF获取。

英文摘要:

Symmetric nonnegative matrix factorization (SymNMF) is a powerful tool for clustering, which typically uses the $k$-nearest neighbor ($k$-NN) method to construct similarity matrix. However, $k$-NN may mislead clustering since the neighbors may belong to different clusters, and its reliability generally decreases as $k$ grows. In this paper, we construct the similarity matrix as a weighted $k$-NN graph with learnable weight that reflects the reliability of each $k$-th NN. This approach reduces the search space of the similarity matrix learning to $n - 1$ dimension, as opposed to the $\mathcal{O}(n^2)$ dimension of existing methods, where $n$ represents the number of samples. Moreover, to obtain a discriminative similarity matrix, we introduce a dissimilarity matrix with a dual structure of the similarity matrix, and propose a new form of orthogonality regularization with discussions on its geometric interpretation and numerical stability. An efficient alternative optimization algorithm is designed to solve the proposed model, with theoretically guarantee that the variables converge to a stationary point that satisfies the KKT conditions. The advantage of the proposed model is demonstrated by the comparison with nine state-of-the-art clustering methods on eight datasets. The code is available at \url{https://github.com/lwl-learning/LSDGSymNMF}.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑