arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~

高阶非凸有限和优化的匹配上下界

Matching Upper and Lower Bounds for Higher-Order Nonconvex Finite-Sum Optimization

Wendao Wu, Haihan Zhang, Chenheng Zhang, Yanyi Li, Chunyuan Zheng, Cong Fang, Haoxuan Li, Zhouchen Lin

arXiv 2609.28202首次发表:更新:

发表机构

Peking University(北京大学)

机构由 AI 辅助整理,请以论文原文为准。

AI 中文总结

本文针对非凸有限和优化,建立了高阶随机预言机复杂度的紧匹配上下界,消除了先前上下界间的√n差距,并给出了包含加性n项的完整复杂度刻画。

AI 中文摘要

我们建立了寻找非凸有限和的一阶驻点的紧随机高阶预言机复杂度。设 n 为分量个数,Δ>0 为初始目标间隙界,L_p>0 为单个 p 阶导数 Lipschitz 界,ε>0 为目标梯度范数。对于每个固定整数 p≥2,返回值和直到 p 阶的所有导数的精确分量查询的极小极大次数,成功概率至少为 2/3,为 Θ_p(n+ΔL_p^{1/p}n^{1-1/(2p)}ε^{-(p+1)/p}),其中常数仅依赖于 p,最坏情况涵盖所有有限维。下界对无限制随机自适应算法成立,并关闭了先前已知的一般阶上下界在 n 依赖上的 √n 差距。我们将稠密弱隐藏扩展到完整的高阶回复,同时保持每个分量的正则性与链长无关。匹配的上界保留了已知的有限和指数,仅需要均方 p 阶导数增量,并通过用精确函数值验证整个递归估计阶段来消除固定置信度的对数损失。该刻画包括每个正参数区域的加性 n 项;它计数具有无限制内部计算的预言机调用。

英文摘要

We establish tight randomized higher-order oracle complexity for finding first-order stationary points of nonconvex finite sums. Let $n$ be the number of components, $Δ>0$ the initial objective-gap bound, $L_p>0$ an individual $p$-th derivative Lipschitz bound, and $ε>0$ the target gradient norm. For every fixed integer $p\ge 2$, the minimax number of exact component queries returning the value and all derivatives through order $p$, with success probability at least $2/3$, is \[ Θ_p\!\left( n+ΔL_p^{1/p}n^{1-1/(2p)} ε^{-(p+1)/p} \right), \] where the constants depend only on $p$ and the worst case ranges over all finite dimensions. The lower bound holds for unrestricted randomized adaptive algorithms and closes the $\sqrt{n}$ gap between the previously known general-order upper and lower bounds in their dependence on $n$. We extend dense weak hiding to complete higher-order replies while keeping each component's regularity independent of the chain length. The matching upper bound retains the known finite-sum exponent, requires only mean-squared $p$-th derivative increments, and removes the fixed-confidence logarithmic loss by verifying entire recursive-estimation epochs with exact function values. The characterization includes the additive $n$ term for every positive parameter regime; it counts oracle calls with unrestricted internal computation.

论文原文

arXiv 摘要页 · PDF 原文 · HTML 原文

↑