arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1408.6686stat.MLcs.LG
收藏

稀疏广义特征值问题的光滑优化

Sparse Generalized Eigenvalue Problem via Smooth Optimization

Hong Kong University of Science and Technology(香港科技大学)

Junxiao Song, Prabhu Babu, Daniel P. Palomar

摘要:

在本文中,我们考虑了广义特征值问题(GEP)的$\ell_{0}$ -范数惩罚形式,旨在提取矩阵对的首稀疏广义特征向量。该公式涉及非凸约束集上的不连续非凹目标函数的最大化,因此在计算上是难以处理的。为了解决这个问题,我们首先近似$\ell_{0}$ -范数的连续代理函数。然后,通过迭代优化代理函数的二次可分函数,在每一次迭代减少到一个规则的广义特征值问题的算法。提出了一种求首广义特征向量的预条件最速上升算法。提出了一种基于平滑的系统方法来处理当使用二次函数来优化不可微代理函数时出现的“奇异性问题”。对于具有特殊结构的稀疏GEPs,推导出了每次迭代都能得到封闭解的算法。数值实验表明,所提出的算法的计算复杂度和支持恢复方面的匹配或优于现有的算法。

英文摘要:

In this paper, we consider an $\ell_{0}$-norm penalized formulation of the generalized eigenvalue problem (GEP), aimed at extracting the leading sparse generalized eigenvector of a matrix pair. The formulation involves maximization of a discontinuous nonconcave objective function over a nonconvex constraint set, and is therefore computationally intractable. To tackle the problem, we first approximate the $\ell_{0}$-norm by a continuous surrogate function. Then an algorithm is developed via iteratively majorizing the surrogate function by a quadratic separable function, which at each iteration reduces to a regular generalized eigenvalue problem. A preconditioned steepest ascent algorithm for finding the leading generalized eigenvector is provided. A systematic way based on smoothing is proposed to deal with the "singularity issue" that arises when a quadratic function is used to majorize the nondifferentiable surrogate function. For sparse GEPs with special structure, algorithms that admit a closed-form solution at every iteration are derived. Numerical experiments show that the proposed algorithms match or outperform existing algorithms in terms of computational complexity and support recovery.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑