AI 中文总结
研究黑箱环境下随机梯度估计，针对有限差分方法中扰动大小最优常数因子难确定问题，提出用少量模拟预算先导估计模型量，使估计器更鲁棒，理论上与最优MSE一阶相同且具竞争力，数值实验验证了其优势。
AI 中文摘要
我们研究了在黑箱环境中的随机梯度估计，其中只能获得函数值的噪声模拟观测。有限差分（FD）方法是此类设置中最广泛使用的零阶梯度估计器之一，通过测量函数值相对于扰动大小的变化来估计梯度。虽然在选择扰动大小相对于模拟预算的最优阶数方面已经有了很好的理解，但最优常数因子依赖于通常未知且被认为与梯度本身一样难以估计的模型特征。因此，FD估计器通常基于对扰动大小的临时调整，这在不同问题实例中可能表现出高度不稳定的性能。在本文中，我们从理论和实践两个角度挑战了这种传统观念。我们表明，通过使用可忽略的模拟预算比例对这些模型量进行先导估计，可以在所得的FD估计器中实现显著的鲁棒性。从理论上讲，我们表明使用由这种先导估计控制的扰动大小已经可以实现与“最优”MSE一阶相同的MSE，就好像最优扰动大小是事先已知的一样。此外，我们展示了这种方法相对于任何规定扰动大小的选择都具有竞争力，即使它们被设计为在合理的目标函数和FD方案类上是极小极大最优的。我们提出的先导估计在实践中易于运行，并且各种数值实验证明了我们的估计器相对于基于临时调整的传统FD方案的鲁棒性和近最优性。
英文摘要
We study stochastic gradient estimation in black-box environments where only noisy simulation observations of function values are available. Finite-difference (FD) methods are among the most widely used zeroth-order gradient estimators in such settings, by measuring the change in function values against a perturbation size. While the optimal order in choosing this perturbation size with respect to the simulation budget is well understood, the optimal constant factor relies on model characteristics that are typically unknown and viewed to be as difficult to estimate as the gradient itself. Consequently, FD estimators are often based on ad hoc tuning of the perturbation size, which may exhibit highly unstable performance across problem instances. In this paper, we challenge this conventional wisdom from both theoretical and practical perspectives. We show that, by pilot-estimating these model quantities using a negligible fraction of the simulation budget, substantial robustness is attained in the resulting FD estimators. Theoretically, we show that using a perturbation size governed by this pilot estimation can already achieve an MSE that is first-order identical to the ``oracle" MSE as if the optimal perturbation size is known in advance. Moreover, we show how such an approach is competitive against any choices of prescribed perturbation size, even if they are designed to be minimax-optimal over reasonable classes of target functions and FD schemes. Our proposed pilot estimation is practically easy to run, and a variety of numerical experiments demonstrate both the robustness and near-oracle optimality of our estimator relative to conventional FD schemes based on ad hoc tuning.
