将检查点更新概率与动量参数联系起来在单循环方差减少方法中
Relating Checkpoint Update Probabilities to Momentum Parameters in Single-Loop Variance Reduction Methods
AI总结：
本文提出了一种单循环方差减少方法，通过将检查点更新概率与动量参数联系起来，实现加速与方差减少的平衡，从而在标准假设下达到近最优复杂性。
摘要：
我们提出了一种单循环方差减少加速框架，该框架将检查点更新概率与动量参数联系起来，用于解决具有光滑部分的复合一般凸问题，其中光滑部分具有有限和结构。在所提出的框架下，动量参数的增长率进一步被改变，从而创建了一个新的连续权衡，即在加速和方差减少之间，由关键参数α∈[0,1]控制。获得了一系列新的复杂性，并在统一框架下重新发现了某些已知方法的复杂性。当由于问题的规模庞大或计算资源不足而限制小批量大小时，通过选择合适的α，可以为任何预设的精度目标实现近最优复杂性。分析显示，尽管考虑的梯度Oracle是精确的，但加速伴随着隐含的方差减少代价，因此获得的最优α不一定对应于最大的允许加速强度。在不预设目标精度的情况下，所提出的方法在标准假设下（n是有限和的组件数）能够达到近最优复杂性~O(n+√n/√ε)，以获得ε精度的解，显著优于之前单循环方差减少方法的最佳复杂性O(n+n/√ε)，后者不超过确定性方法FISTA的复杂性。数值实验展示了所提出方法的效率，并验证了其他理论发现。
英文摘要：
We propose a single-loop variance-reduced acceleration framework, which relates checkpoint update probabilities to momentum parameters, for solving the composite general convex problem where the smooth part has the finite-sum structure. Under the proposed framework, the growth rate of the momentum parameter is further altered, creating a novel continuous trade-off between acceleration and variance reduction, controlled by the key parameter $α\in [0,1]$. A series of novel complexity is obtained, and some complexity of distinct known methods are rediscovered under the unified framework. When the mini-batch size is restricted due to the massive scale of the problem or the computational resource shortage, near-optimal complexity can still be achieved by choosing suitable $α$ for any prefixed target accuracy. Analysis shows that although the considered gradient oracle is exact, acceleration comes with implicit price of heavier variance reduction, hence the obtained optimal $α$ not necessarily corresponds to the largest allowable acceleration strength. Without prefixing the target accuracy, the proposed method achieves the near-optimal complexity $\tilde{\mathcal{O}}(n+\sqrt{n}/\sqrtε)$ to obtain an $ε$-accurate solution under standard assumptions ($n$ is the number of components of the finite-sum), significantly improves upon previous best complexity $\mathcal{O}(n+n/\sqrtε)$ of single-loop variance reduction methods, which does not exceed the complexity of the deterministic method FISTA. Numerical experiments demonstrate the efficiency of the proposed method and validate other theoretical findings.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
