局部强凸总体风险的超额风险特征
Characterization of Excess Risk for Locally Strongly Convex Population Risk
摘要：
我们建立了适当的迭代算法训练的模型，近似局部极小值的期望超额风险的上界。与建立在强全局强凸性或全局增长条件上的结果不同，PL-不等式，我们只要求人口风险是\emph{locally}强凸围绕其局部极小值。具体地说，我们在凸问题下的界是$\tilde{\cO}(1/n)$阶的。对于具有$d$模型参数的非凸问题，使得$d/n$小于与$n$无关的阈值，如果经验风险没有具有高概率的伪局部最小值，则可以保持$\tilde{\cO}(1/n)$的阶。此外，非凸问题的界成为$\tilde{\cO}(1/\sqrt{n})$没有这样的假设。我们的结果是通过算法的稳定性和表征的经验风险的景观。与已有的基于算法稳定性的结果相比，我们的界是维数不敏感的，并且对算法的实现、学习率和迭代次数没有限制。我们的边界强调，与局部强凸的人口风险，任何适当的迭代算法训练的模型可以推广，甚至对非凸问题，和$d$是大的。
英文摘要：
We establish upper bounds for the expected excess risk of models trained by proper iterative algorithms which approximate the local minima. Unlike the results built upon the strong globally strongly convexity or global growth conditions e.g., PL-inequality, we only require the population risk to be \emph{locally} strongly convex around its local minima. Concretely, our bound under convex problems is of order $\tilde{\cO}(1/n)$. For non-convex problems with $d$ model parameters such that $d/n$ is smaller than a threshold independent of $n$, the order of $\tilde{\cO}(1/n)$ can be maintained if the empirical risk has no spurious local minima with high probability. Moreover, the bound for non-convex problem becomes $\tilde{\cO}(1/\sqrt{n})$ without such assumption. Our results are derived via algorithmic stability and characterization of the empirical risk's landscape. Compared with the existing algorithmic stability based results, our bounds are dimensional insensitive and without restrictions on the algorithm's implementation, learning rate, and the number of iterations. Our bounds underscore that with locally strongly convex population risk, the models trained by any proper iterative algorithm can generalize well, even for non-convex problems, and $d$ is large.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
