arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1907.13307math.OCcs.LGstat.ML
收藏

随机凸优化中从低概率到高置信度

From low probability to high confidence in stochastic convex optimization

Cornell University(康奈尔大学);Microsoft Research(微软研究院);University of Minnesota(明尼苏达大学)

Damek Davis, Dmitriy Drusvyatskiy, Lin Xiao, Junyu Zhang

摘要:

随机凸优化中的标准结果限定了算法生成期望函数值较小的点所需的样本个数。更细致入微的高概率保证是罕见的,通常依赖于“轻尾”噪声假设或表现出更差的样本复杂性。在这项工作中,我们证明了一类广泛的强凸问题的随机最优化算法,可以增加高的置信区间的开销成本,只有对数的置信水平和polylogarithmic的条件数。我们提出的程序,称为EST-Boost,是基本的,并建立在两个众所周知的成分:鲁棒的距离估计和邻近点方法。我们讨论的后果流(在线)算法和离线算法的经验风险最小化的基础上。

英文摘要:

Standard results in stochastic convex optimization bound the number of samples that an algorithm needs to generate a point with small function value in expectation. More nuanced high probability guarantees are rare, and typically either rely on "light-tail" noise assumptions or exhibit worse sample complexity. In this work, we show that a wide class of stochastic optimization algorithms for strongly convex problems can be augmented with high confidence bounds at an overhead cost that is only logarithmic in the confidence level and polylogarithmic in the condition number. The procedure we propose, called proxBoost, is elementary and builds on two well-known ingredients: robust distance estimation and the proximal point method. We discuss consequences for both streaming (online) algorithms and offline algorithms based on empirical risk minimization.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑