轻松改进自适应零阶优化
Refining Adaptive Zeroth-Order Optimization at Ease
AI总结：
本文提出 R-AdaZO，通过利用一阶矩的方差缩减效应并改进二阶矩估计，构建方差感知收敛框架，在多种零阶任务中实现更快收敛。
摘要：
近年来，零阶（ZO）优化在梯度信息不可获取或代价过高的场景中发挥着关键作用，例如黑盒系统和资源受限环境。尽管 ZO-AdaMM 等现有自适应方法已展现出良好前景，但它们在优化过程中对矩信息利用不足这一根本限制，通常会导致收敛性能不佳。为克服这些限制，本文提出了 Refined Adaptive Zeroth-Order Optimization（R-AdaZO，改进型自适应零阶优化）。具体而言，我们首先揭示了一阶矩估计在 ZO 梯度估计中尚未被开发的方差缩减效应，该效应提升了 ZO 更新的准确性与稳定性。随后，我们基于这些经方差缩减的梯度估计改进二阶矩估计，以更好地捕捉优化景观的几何结构，从而实现对 ZO 更新更有效的缩放。我们给出了严格的理论分析，表明：（a）这是首次对 ZO 优化中一阶矩估计的方差缩减进行分析；（b）改进后的二阶矩估计能够更准确地逼近其无方差理想形式；（c）首次提出面向自适应 ZO 方法的方差感知收敛框架，该框架本身可能具有独立研究价值；（d）R-AdaZO 相比 ZO-AdaMM 等现有基线具有更快的收敛速度。我们开展了大量实验，包括合成问题、黑盒对抗攻击以及大语言模型（LLM）的内存高效微调，进一步验证了 R-AdaZO 优越的收敛性能，表明 R-AdaZO 为现实世界中的 ZO 优化挑战提供了一种改进方案。
英文摘要：
Recently, zeroth-order (ZO) optimization plays an essential role in scenarios where gradient information is inaccessible or unaffordable, such as black-box systems and resource-constrained environments. While existing adaptive methods such as ZO-AdaMM have shown promise, they are fundamentally limited by their underutilization of moment information during optimization, usually resulting in underperforming convergence. To overcome these limitations, this paper introduces Refined Adaptive Zeroth-Order Optimization (R-AdaZO). Specifically, we first show the untapped variance reduction effect of first moment estimate on ZO gradient estimation, which improves the accuracy and stability of ZO updates. We then refine the second moment estimate based on these variance-reduced gradient estimates to better capture the geometry of the optimization landscape, enabling a more effective scaling of ZO updates. We present rigorous theoretical analysis to show (a) the first analysis to the variance reduction of first moment estimate in ZO optimization, (b) the improved second moment estimates with a more accurate approximation of its variance-free ideal, (c) the first variance-aware convergence framework for adaptive ZO methods, which may be of independent interest, and (d) the faster convergence of R-AdaZO than existing baselines like ZO-AdaMM. Our extensive experiments, including synthetic problems, black-box adversarial attack, and memory-efficient fine-tuning of large language models (LLMs), further verify the superior convergence of R-AdaZO, indicating that R-AdaZO offers an improved solution for real-world ZO optimization challenges.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
