发表机构
University of Toronto; Vector Institute; University of Hong Kong（多伦多大学; 向量研究所; 香港大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
针对非凸零阶优化，提出控制变量零阶下降（CV-ZOD）框架，自适应利用方向提示，实现介于一阶与零阶之间的收敛速率，并在模拟科学优化任务中验证有效性。
AI 中文摘要
我们研究在方向提示的辅助下对非凸函数进行零阶优化，方向提示是真实梯度方向的廉价但可能不准确的近似，由每次迭代时的线性子空间给出。为了自适应地利用这些提示，同时保持对其质量的鲁棒性，我们引入了控制变量零阶下降（CV-ZOD），这是一个新框架，通过一个可根据方向提示设置的控制变量来改进经典的零阶梯度估计器。我们首先证明，在每次迭代中最优地设置参考向量和步长的oracle算法，其收敛速度在基于轨迹上提示质量的一阶$O(1/T)$速率和零阶$O(d/T)$速率之间插值。然后，我们开发了CV-ZOD的一个实用变体，该变体在没有任何提示质量先验知识的情况下，实现了与oracle相同的保证（相差对数因子）。我们在基于模拟的科学优化任务上实证验证了该方法，展示了在非凸景观上的持续进展，在这些景观中，零阶下降较慢，而现有的引导方法在引导质量下降时会停滞不前。
英文摘要
We study zeroth-order optimization of non-convex functions with the aid of directional hints, which are cheap but potentially inaccurate approximations of the true gradient direction, given by linear subspaces at each iteration. To leverage these hints adaptively while maintaining robustness to their quality, we introduce Control-Variate Zeroth-Order Descent (CV-ZOD), a new framework that refines the classical zeroth-order gradient estimator with a control variate that can be set based on the directional hints. We first show that the oracle algorithm that optimally sets the reference vector and step size at each iteration achieves a convergence rate that interpolates between the first-order $O(1/T)$ rate and the zeroth-order $O(d/T)$ rate, depending on the quality of the hints along the trajectory. We then develop a practical variant of CV-ZOD that achieves the same oracle guarantee up to logarithmic factors, without any prior knowledge of the hint quality. We validate the method empirically on simulation-based scientific optimization tasks, demonstrating sustained progress on non-convex landscapes where zeroth-order descent is slower and existing guided methods stall as guidance deteriorates.
