arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2405.20116math.OCcs.NAmath.NAmath.PR
收藏

零阶和一阶随机信赖域算法的复杂度

Complexity of Zeroth- and First-order Stochastic Trust-Region Algorithms

Yunsoo Ha, Sara Shashaani, Raghu Pasupathy

AI总结:

本研究探讨在随机信赖域算法的模型更新和候选评估步骤中引入公共随机数对样本复杂度的影响,发现其效果取决于样本路径平滑性和预言机阶数,并显著改善一阶平滑情形下的复杂度。

摘要:

模型更新(MU)和候选评估(CE)是许多随机信赖域(TR)算法中纳入的经典步骤。这些步骤中投入的采样工作量,通常以控制模型误差为目标来决定,在很大程度上决定了随机TR算法的样本复杂度。鉴于MU和CE适用于方差缩减,我们研究了在MU和CE中引入公共随机数(CRN)对复杂度的影响。使用ASTRO和ASTRO-DF作为一阶和零阶算法族的原型,我们证明CRN的有效性导致了一系列复杂度,这些复杂度取决于样本路径的规律性和预言机阶数。例如,我们发现在具有平滑样本路径的一阶预言机设置中,CRN的效果显著——带有CRN的ASTRO实现了几乎必然(a.s.)的$\ ilde{O}(ε^{-2})$样本复杂度,而通用无CRN设置为a.s.的$\ ilde{O}(ε^{-6})$。相比之下,当样本路径非Lipschitz时,CRN的效果减弱,样本复杂度在零阶和一阶设置中分别从a.s.的$\ ilde{O}(ε^{-6})$改善到a.s.的$\ ilde{O}(ε^{-5})$和$\ ilde{O}(ε^{-4})$。由于我们的结果意味着复杂度的改善在很大程度上继承自方差缩减的通用方面,例如零阶设置中的有限差分和一阶设置中MU内的样本路径平滑性,我们预计在其他情境中也会有类似的趋势。

英文摘要:

Model update (MU) and candidate evaluation (CE) are classical steps incorporated inside many stochastic trust-region (TR) algorithms. The sampling effort exerted within these steps, often decided with the aim of controlling model error, largely determines a stochastic TR algorithm's sample complexity. Given that MU and CE are amenable to variance reduction, we investigate the effect of incorporating common random numbers (CRN) within MU and CE on complexity. Using ASTRO and ASTRO-DF as prototype first-order and zeroth-order families of algorithms, we demonstrate that CRN's effectiveness leads to a range of complexities depending on sample-path regularity and the oracle order. For instance, we find that in first-order oracle settings with smooth sample paths, CRN's effect is pronounced -- ASTRO with CRN achieves $\tilde{O}(ε^{-2})$ a.s. sample complexity compared to $\tilde{O}(ε^{-6})$ a.s. in the generic no-CRN setting. By contrast, CRN's effect is muted when the sample paths are not Lipschitz, with the sample complexity improving from $\tilde{O}(ε^{-6})$ a.s. to $\tilde{O}(ε^{-5})$ and $\tilde{O}(ε^{-4})$ a.s. in the zeroth- and first-order settings, respectively. Since our results imply that improvements in complexity are largely inherited from generic aspects of variance reduction, e.g., finite-differencing for zeroth-order settings and sample-path smoothness for first-order settings within MU, we anticipate similar trends in other contexts.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑