arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1507.04734math.OCcs.LGstat.ML
收藏

变分格拉姆函数:凸分析与Optimization

Variational Gram Functions: Convex Analysis and Optimization

University of Washington(华盛顿大学);Microsoft Research(微软研究院)

Amin Jalali, Maryam Fazel, Lin Xiao

摘要:

我们提出了一类新的凸罚函数,称为\emph{variational Gram functions}(VGF),它可以促进向量空间中一组向量之间的成对关系,如正交性。这些函数可以作为正则化的凸优化问题所产生的分层分类,多任务学习,估计向量与不相交的支持,以及其他应用。研究了VGF的凸性,给出了VGF的凸共轭、次微分和邻近算子的有效刻画。我们讨论有效的优化算法正则化损失最小化问题的损失承认一个共同的,但简单的,变分表示和正则化是一个VGF。这些算法享有一个简单的核技巧,一个有效的线搜索,以及计算上的优势,一阶方法的基础上的次微分或邻近的地图。我们还建立了这类学习问题的一般表示定理。最后,一个层次分类问题的数值实验,以证明VGF和相关的优化算法的有效性。

英文摘要:

We propose a new class of convex penalty functions, called \emph{variational Gram functions} (VGFs), that can promote pairwise relations, such as orthogonality, among a set of vectors in a vector space. These functions can serve as regularizers in convex optimization problems arising from hierarchical classification, multitask learning, and estimating vectors with disjoint supports, among other applications. We study convexity for VGFs, and give efficient characterizations for their convex conjugates, subdifferentials, and proximal operators. We discuss efficient optimization algorithms for regularized loss minimization problems where the loss admits a common, yet simple, variational representation and the regularizer is a VGF. These algorithms enjoy a simple kernel trick, an efficient line search, as well as computational advantages over first order methods based on the subdifferential or proximal maps. We also establish a general representer theorem for such learning problems. Lastly, numerical experiments on a hierarchical classification problem are presented to demonstrate the effectiveness of VGFs and the associated optimization algorithms.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑