一种用于大规模机器学习的近似子采样近端牛顿型方法
An inexact subsampled proximal Newton-type method for large-scale machine learning
AI总结：
本文提出一种快速近端牛顿型算法，通过子采样构造牛顿子问题，提升大规模优化效率，实验验证其在ℓ₁正则化逻辑回归中的优越性。
摘要：
我们提出了一种快速的近端牛顿型算法，用于最小化带有正则化的有限和。该算法能够在$\tilde{\mathcal{O}}(d(n + \sqrt{κd})\log(\frac{1}ε))$ FLOPS内返回一个$ε$-次优解，其中$n$是样本数，$d$是特征维度，$κ$是条件数。只要$n > d$，该方法比最先进的加速随机一阶方法更高效，后者需要$\tilde{\mathcal{O}}(d(n + \sqrt{κn})\log(\frac{1}ε))$ FLOPS。关键思想是通过子采样构造牛顿子问题，以保持目标函数的有限和结构，从而利用最近的随机一阶方法进展来求解子问题。实验结果验证了所提算法在真实数据集上的ℓ₁正则化逻辑回归任务中优于先前算法。
英文摘要：
We propose a fast proximal Newton-type algorithm for minimizing regularized finite sums that returns an $ε$-suboptimal point in $\tilde{\mathcal{O}}(d(n + \sqrt{κd})\log(\frac{1}ε))$ FLOPS, where $n$ is number of samples, $d$ is feature dimension, and $κ$ is the condition number. As long as $n > d$, the proposed method is more efficient than state-of-the-art accelerated stochastic first-order methods for non-smooth regularizers which requires $\tilde{\mathcal{O}}(d(n + \sqrt{κn})\log(\frac{1}ε))$ FLOPS. The key idea is to form the subsampled Newton subproblem in a way that preserves the finite sum structure of the objective, thereby allowing us to leverage recent developments in stochastic first-order methods to solve the subproblem. Experimental results verify that the proposed algorithm outperforms previous algorithms for $\ell_1$-regularized logistic regression on real datasets.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
