arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2602.00075cs.LGcs.MS
收藏

通过模拟实现的维度窥视:用于零阶离散优化中低方差梯度的方法

Dimensional Peeking for Low-Variance Gradients in Zeroth-Order Discrete Optimization via Simulation

Nanyang Technological University(南洋理工大学)

Philipp Andelfinger, Wentong Cai

AI总结:

本文提出一种通过模拟实现的维度窥视方法,用于减少零阶离散优化中的梯度方差,提升优化性能。

摘要:

基于梯度的优化方法常用于在高维空间中寻找局部极值。当无法直接评估导数时,随机估计器可以提供近似梯度。然而,这些估计器基于扰动的函数采样会引入方差,可能导致收敛缓慢。在本文中,我们提出了维度窥视,一种通过模拟实现的梯度估计方差减少方法,用于离散优化。通过将采样粒度从标量值提升到遵循相同控制流路径的值类别,我们提高了每个模拟评估所获取的信息量。我们的推导表明,该方法不引入任何偏差。我们通过自定义数值数据类型实现了该方法,以透明地在C++程序上执行维度窥视。对于三个基于模拟的优化问题,输入维度高时,观察到方差减少高达7.9。与三种元启发式方法的优化进展相比,维度窥视增强了零阶优化在离散和非凸模拟中的竞争力。

英文摘要:

Gradient-based optimization methods are commonly used to identify local optima in high-dimensional spaces. When derivatives cannot be evaluated directly, stochastic estimators can provide approximate gradients. However, these estimators' perturbation-based sampling of the objective function introduces variance that can lead to slow convergence. In this paper, we present dimensional peeking, a variance reduction method for gradient estimation in discrete optimization via simulation. By lifting the sampling granularity from scalar values to classes of values that follow the same control flow path, we increase the information gathered per simulation evaluation. Our derivation from an established smoothed gradient estimator shows that the method does not introduce any bias. We present an implementation via a custom numerical data type to transparently carry out dimensional peeking over C++ programs. Variance reductions by factors of up to 7.9 are observed for three simulation-based optimization problems with high-dimensional input. The optimization progress compared to three meta-heuristics shows that dimensional peeking increases the competitiveness of zeroth-order optimization for discrete and non-convex simulations.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑