一种无参数零阶方法：协方差矩阵自适应与有效维度

中文摘要 AI 辅助

零阶优化方法对于求解梯度信息不可用或计算代价高昂的黑箱问题至关重要。本文提出POEM-CMA，一种新颖的无参数随机零阶算法，通过整合协方差矩阵对齐与有效维度的概念，扩展了近期提出的POEM方法。与依赖各向同性随机方向的传统零阶方法不同，POEM-CMA通过从梯度估计构建协方差矩阵来进行各向异性采样，从而使算法能够将采样重点集中在最具信息量的方向上。我们引入了经验有效维度 $d^* = \frac{\operatorname{tr}(\hat{\Sigma})}{\lambda_{\max}(\hat{\Sigma})}$ 的使用，该维度反映了问题的内在维度，并在采样和复杂度分析中取代了环境维度。我们证明POEM-CMA达到了接近最优的收敛速率，仅需 $\tilde{\mathcal{O}}\left(\frac{d^* \kappa(\hat{\Sigma}) L^2 D_{\mathcal{X}}^2}{\varepsilon^2}\right)$ 次随机零阶预言机查询。该方法保持完全无参数，并在具有低秩结构（即 $d^* \ll d$）的问题上展现出相对于原始POEM的显著改进。使用LibSVM数据集进行的铰链损失二分类数值实验证实了所提方法的实际优越性。

英文摘要

Zeroth-order optimization methods are essential for solving black-box problems where gradient information is unavailable or expensive to compute. This paper presents POEM-ES, a novel parameter-free stochastic zeroth-order algorithm that extends the recent POEM method by integrating subspace preconditioning with ellipsoidal randomized sampling. In contrast to traditional zeroth-order approaches that rely on isotropic random directions, POEM-ES performs anisotropic sampling guided by a fixed structural symmetric positive semi-definite (SPSD) preconditioner $\hatΣ$ that encodes the underlying low-dimensional geometry. Under a standard structural spectral normalization where $λ_{\max}(\hatΣ) = 1$, we introduce the use of the empirical effective dimension $d^* = \operatorname{tr}(\hatΣ)$, which reflects the intrinsic dimensionality of the problem and guides both the sampling and randomized smoothing parameter schedules. In practice, such a preconditioner can be effectively obtained via pilot sampling, historical trajectories, or domain-specific expert knowledge. We prove that POEM-ES achieves a dimension-reduced convergence rate under low-rank structural assumptions, requiring only $\tilde{\mathcal{O}}\left( \frac{\left( r^2 κ(\hatΣ) + d^* \right) L^2 D_{\mathcal{X}}^2}{\varepsilon^2} \right)$ stochastic zeroth-order oracle queries. The method remains fully parameter-free and demonstrates significant improvements over the original POEM in problems with low-rank structure where $d^* \ll d$. Numerical experiments on hinge-loss binary classification tasks using LibSVM datasets confirm the practical superiority of the proposed approach.

发表机构

• Moscow Institute of Physics and Technology（莫斯科物理技术学院）

机构由 AI 辅助整理，请以论文原文为准。