发表机构
University of Maryland, College Park（马里兰大学学院公园分校）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
本文证明在线性可分数据上，逻辑回归的大步长梯度下降在任意维度下达到损失ε的步数为多对数复杂度，通过划分振荡阶段为嵌套区间，利用间隔和秩限制深度，显著改进了现有收敛速率。
AI 中文摘要
我们研究了在线性可分数据上对逻辑回归使用大常数步长的梯度下降（GD）方法。现有分析表明，在激进步长下，达到损失ε的加速率为Õ(1/√ε)，尽管损失可能最初会振荡。对振荡动力学的更严格控制仅适用于二维数据。我们证明了在任意维度上显著更快的速率：使用大步长η=1/ε的GD在O(ln^p(1/ε))步内达到损失ε，其中p仅依赖于数据的间隔（margin）和秩（rank）。我们的证明改进了GD从振荡阶段到稳定阶段的过渡时间的界限，之后损失单调递减。我们将振荡阶段划分为递归嵌套的区间。间隔和秩限制了嵌套深度，计数论证限制了每个深度上的区间数量，共同产生了多对数步复杂度。
英文摘要
We study gradient descent (GD) with a large constant stepsize for logistic regression on linearly separable data. Existing analysis shows an accelerated rate of $\widetilde{O}(1/\sqrtε)$ to reach loss $ε$ with an aggressive stepsize, although the loss may initially oscillate. Tighter control of the oscillatory dynamics has been available only for two-dimensional data. We prove a substantially faster rate in arbitrary dimension: GD with a large stepsize $η=1/ε$ reaches loss $ε$ within $O(\ln^{p}(1/ε))$ steps, where $p$ depends only on the margin and the rank of the data. Our proof improves the bound on the transition time of GD from the oscillatory to the stable phase, after which the loss decreases monotonically. We split the oscillatory phase into recursively nested intervals. The margin and the rank bound the nesting depth, and a counting argument bounds the number of intervals at each depth, together yielding the polylogarithmic step complexity.
