NEPv方法用于带$(2,1)$-范数正则化的Stiefel流形优化

中文摘要 AI 辅助

行稀疏投影在机器学习（ML）中是一种有用的工具，例如在特征选择中，旨在为各种ML目标选择最相关的特征。寻求高质量行稀疏投影的一种方法是将ML目标（如PCA、LDA和OCCA的目标）与非光滑的矩阵$(2,1)$-范数正则化相结合。这种组合导致了Stiefel流形上具有挑战性的优化问题，需要高效求解。本文建立了一个统一的NEPv框架，以高效处理带$(2,1)$-范数正则化的Stiefel流形优化。同时研究了$(2,1)$-范数正则化的效果。通过将当今数据科学应用中常见的学习目标与$(2,1)$-范数正则化相结合，展示了该框架的广泛适用性。数值实验说明了NEPv方法的使用，并深入了解了在实际应用中合适的正则化参数应如何选取。

英文摘要

Row-sparse projection provides a useful tool in machine learning (ML) when it comes to, for example, feature selection, aiming to choose most relevant features for various ML objectives. One way to seek a high quality row-sparse projection is to combine an ML objective, such as the ones for PCA, LDA, and OCCA, with the matrix $(2,1)$-norm regularization which is nonsmooth. Such combinations result in challenging optimization problems on the Stiefel manifold that need to be solved efficiently. In this paper, a unifying NEPv framework is established to efficiently deal with optimization on the Stiefel manifold with the $(2,1)$-norm regularization. The effect of the $(2,1)$-norm regularization is also investigated. The wide applicability of the framework is demonstrated through the combinations of common learning objectives in today's data science applications with the $(2,1)$-norm regularization. Numerical experiments are presented to illustrate the use of the NEPv approach and to gain insights as to what a proper regularizing parameter should have in real-world applications.

发表机构

• University of Texas at Arlington（阿灵顿得克萨斯大学）
• Soochow University（苏州大学）
• University of California, Davis（加州大学戴维斯分校）

机构由 AI 辅助整理，请以论文原文为准。