发表机构
Institute of Industrial Science, The University of Tokyo; Universal Biology Institute, The University of Tokyo（东京大学工业科学研究所; 东京大学综合生物学院）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
该研究针对输入不确定性优化（OIU）问题，提出基于Rao-Blackwell化梯度估计器的表型加速进化策略（PAES），通过利用可观测的已实现输入信息降低梯度估计方差，实验显示其收敛速度快于常规进化策略。
AI 中文摘要
我们研究输入不确定性下的优化（OIU），即目标函数的输入而非目标函数本身存在不确定性。OIU出现在具有生产公差的制造过程、具有执行噪声的物理系统控制、混合专家模型以及强化学习（RL）中。现有大多数方法通过使用目标函数值求解OIU，但会丢弃已实现输入的信息，即便在多种应用中已实现输入是可观测的。此处的问题是，被丢弃的已实现输入信息是否有助于加速优化过程。我们针对进化策略（ES）给出肯定回答，通过理论证明，已实现输入的信息可通过Rao-Blackwell化降低梯度估计器的方差。我们利用该Rao-Blackwell化梯度估计器，提出表型加速进化策略（PAES），这是针对OIU的ES改进方法。数值实验表明，在从简单连续优化问题到RL基准测试的任务中，PAES的收敛速度快于常规ES。
英文摘要
We investigate Optimization under Input Uncertainty (OIU), in which the input to the objective function, rather than the objective function itself, is subject to uncertainty. OIU appears in manufacturing processes with production tolerance, control of physical systems with actuation noise, Mixture of Experts, and Reinforcement Learning (RL). Most of the existing approaches solve OIU by using the value of the objective function but discard the information of the realized input, even though the realized input is observable in various applications. The question here is whether the discarded information of the realized input is useful to accelerate the optimization process. We affirmatively answer this question for Evolutionary Strategy (ES) by theoretically showing that the information of the realized input can reduce the variance of the gradient estimator via Rao-Blackwellization. Using the Rao-Blackwellized gradient estimator, we propose Phenotype-Accelerated Evolutionary Strategy (PAES), which is a refinement of ES for OIU. Numerical experiments show that PAES converges faster than the usual ES from simple continuous optimization problems to RL benchmarks.
Comments29 pages
