发表机构
Institute of Artificial Intelligence and Future Networks, Faculty of Arts and Sciences, Beijing Normal University; Guangdong Key Lab of AI and Multi-Modal Data Processing, BNU-HKBU United International College（北京师范大学艺术与科学学院人工智能与未来网络研究所; 北师大-香港浸会大学联合国际学院广东省人工智能与多模态数据处理重点实验室）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
针对非凸和PL有限和优化，提出稠密弱隐藏构造，证明匹配下界并设计Restarted PAGE算法，闭合个体光滑性下的复杂度差距。
AI 中文摘要
在个体光滑性假设下，非凸有限和优化的最优增量一阶预言机（IFO）复杂度问题尚未解决。已知算法需要$O(n+\sqrt n\\,\Delta L_{\max}/\varepsilon^2)$次调用，而现有下界在第二项中缺少因子$\sqrt n$。我们证明了随机IFO算法的匹配下界$\Omega(n+\sqrt n\\,\Delta L_{\max}/\varepsilon^2)$，该下界适用于包括从完整先前历史中选择分量索引和查询点的算法。因此，在个体光滑性和均方光滑性下，PAGE和SPIDER在通用常数意义下达到极小极大最优。在全局Polyak--Lojasiewicz（PL）条件下，我们利用类似思想获得了大$\kappa_{\max}$情形下的$\Omega(n+\kappa_{\max}\sqrt n\log(\Delta/\varepsilon))$下界。现有PL下界仅关注相对较大的$\kappa$。对于先前未探索的$\kappa_{\max}<\sqrt n$区域，我们发现了一个新速率$\Omega(n+n\log(\Delta/\varepsilon)/(1+\log(\sqrt n/\kappa_{\max})))$。该下界激励了我们提出的Restarted PAGE算法，其上限在小$\kappa_{\max}$时匹配新速率，在大$\kappa_{\max}$时恢复标准PAGE速率，表明两个界都几乎紧。我们的下界使用了提出的稠密弱隐藏构造。该构造将每个隐藏方向分散到所有分量中，因此单个IFO查询仅揭示弱信息，而完整平均值保留方向。未揭示的阶段即使对任意查询点也保持不活跃，迫使在取得实质进展前需要多次调用才能暴露一个阶段。平衡这种揭示成本与个体光滑性允许的阶段数，产生了缺失的$\sqrt n$因子。
英文摘要
Under individual smoothness, the optimal incremental first-order oracle (IFO) complexity of nonconvex finite-sum optimization is open. Known algorithms use $O(n+\sqrt n\,ΔL_{\max}/\varepsilon^2)$ calls, while existing lower bounds miss the factor $\sqrt n$ in the second term. We prove the matching lower bound $Ω(n+\sqrt n\,ΔL_{\max}/\varepsilon^2)$ for randomized IFO algorithms, including those that choose component indices and query points from the full preceding history. Thus PAGE and SPIDER are minimax optimal up to universal constants under individual and mean-squared smoothness. Under the global Polyak--Lojasiewicz (PL) condition, we use a similar idea to obtain an $Ω(n+κ_{\max}\sqrt n\log(Δ/\varepsilon))$ lower bound for large $κ_{\max}$. Existing PL lower bounds have focused only on relatively large $κ$. For the previously unexplored regime $κ_{\max}<\sqrt n$, we discover a new rate, $Ω(n+n\log(Δ/\varepsilon)/(1+\log(\sqrt n/κ_{\max})))$. This lower bound motivates our Restarted PAGE algorithm, whose upper bound matches the new rate for small $κ_{\max}$ and recovers the standard PAGE rate for large $κ_{\max}$, showing that both bounds are nearly tight. Our lower bounds use the proposed dense weak hiding construction. It spreads each hidden direction across all components, so an individual IFO query reveals only weak information while the full average preserves the direction. Unrevealed stages remain inactive even for arbitrary query points, forcing many calls to expose a stage before substantial progress is possible. Balancing this revelation cost against the number of stages allowed by individual smoothness yields the missing $\sqrt n$ factor.
Comments55 pages, 7 figures
