arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2106.02018cs.LGstat.ML
收藏

基于径向基函数分量的非线性矩阵逼近

Nonlinear Matrix Approximation with Radial Basis Function Components

Lawrence Berkeley National Laboratory(劳伦斯伯克利国家实验室)

Elizaveta Rebrova, Yu-Hang Tang

摘要:

我们介绍和调查矩阵近似分解成径向基函数(RBF)组件的总和。RBF分量是一对向量之间的外积的推广,其中RBF函数取代了各个向量元素之间的标量乘法。尽管RBF函数是正定的,但分量之间的求和并不限于凸组合,并且允许我们计算任何不一定对称或正定的实矩阵的分解。我们制定的问题,寻求这样的分解作为一个非线性和非凸损失函数的优化问题。几个现代版本的梯度下降方法,包括其可扩展的随机对应物,被用来解决这个问题。我们提供了广泛的经验证据的有效性的RBF分解和基于梯度的拟合算法。虽然在概念上受到奇异值分解(SVD)的启发,但我们提出的非线性对应物通过大幅减少近似具有相同L2误差的数据矩阵所需的内存来大幅降低SVD。例如,对于高斯噪声、图形邻接矩阵和核矩阵,它可以节省2到6倍的内存。此外,这种基于邻近度的分解可以在涉及以下的应用中提供附加的可解释性,例如,捕捉数据内部的低维结构,保持图的连通性结构,保持图像的锐度。

英文摘要:

We introduce and investigate matrix approximation by decomposition into a sum of radial basis function (RBF) components. An RBF component is a generalization of the outer product between a pair of vectors, where an RBF function replaces the scalar multiplication between individual vector elements. Even though the RBF functions are positive definite, the summation across components is not restricted to convex combinations and allows us to compute the decomposition for any real matrix that is not necessarily symmetric or positive definite. We formulate the problem of seeking such a decomposition as an optimization problem with a nonlinear and non-convex loss function. Several modern versions of the gradient descent method, including their scalable stochastic counterparts, are used to solve this problem. We provide extensive empirical evidence of the effectiveness of the RBF decomposition and that of the gradient-based fitting algorithm. While being conceptually motivated by singular value decomposition (SVD), our proposed nonlinear counterpart outperforms SVD by drastically reducing the memory required to approximate a data matrix with the same L2 error for a wide range of matrix types. For example, it leads to 2 to 6 times memory save for Gaussian noise, graph adjacency matrices, and kernel matrices. Moreover, this proximity-based decomposition can offer additional interpretability in applications that involve, e.g., capturing the inner low-dimensional structure of the data, retaining graph connectivity structure, and preserving the acutance of images.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑