发表机构
Tsinghua University; Stanford University（清华大学; 斯坦福大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
本文否证了逻辑回归在稳定性边缘过渡时间与步长无关的猜想，证明最坏情况过渡时间为$(\log\eta)^{\min\{n-2,d-2\}}$量级，并通过归纳控制样本变化构造匹配实例。
AI 中文摘要
我们研究在线性可分数据上，使用大步长常数$\eta$的梯度下降进行逻辑回归。此类动力学可能表现出特征性的稳定性边缘现象，其中损失最初振荡，然后过渡到单调下降的稳定阶段。现有工作在维度$d=2$且$\eta \to \infty$时提供了紧致的$\Theta(1)$界，并推测在任意维度$d\geq 2$下存在与$\eta$无关的界。在本文中，我们通过证明对于每个固定样本量$n\geq 2$和足够小的间隔$\gamma$，最坏情况下的过渡时间为$$\Theta\\!\left((\log\eta)^{\min\{n-2,d-2\}}\right)$$，且对$d\geq2$一致成立，从而否定了这一猜想。建立紧致界的关键挑战在于，对梯度贡献最大的样本可能在迭代中反复变化。为解决此问题，我们通过维度和样本量的归纳来控制这种变化，并构造匹配的困难实例。
英文摘要
We study logistic regression on linearly separable data under gradient descent with a large constant stepsize $η$. Such dynamics may exhibit a characteristic Edge of Stability phenomenon, in which the loss initially oscillates before transitioning to a stable phase of monotone decrease. Existing work provides a tight $Θ(1)$ bound in dimension $d=2$ as $η\to \infty$ and conjectures a bound independent of $η$ in arbitrary dimensions $d\geq 2$. In this paper, we disprove this conjecture by showing that, for every fixed sample size $n\geq 2$ and sufficiently small margin $γ$, the worst-case transition time is $$Θ\!\left((\logη)^{\min\{n-2,d-2\}}\right)$$ uniformly over $d\geq2$. The key challenge in establishing a tight bound is that the sample contributing most strongly to the gradient can change repeatedly across iterations. To address this issue, we control such changes by induction on dimension and sample size, and construct matching hard instances.
