利用传感器数据改进梯度估计
Improving Gradient Estimation by Incorporating Sensor Data
摘要:
一个有效的策略搜索算法应该从尽可能少的试验中估计目标函数相对于策略参数的局部梯度。虽然大多数策略搜索方法通过观察策略试验期间获得的奖励来估计此梯度,但我们从理论和经验上证明,考虑到传感器数据,也可以提供更好的梯度估计,从而更快地学习。原因是,由于环境中的噪声,在策略执行期间获得的奖励因试验而异;与噪声相关的传感器数据可用于部分校正这种变化,从而产生具有较低方差的估计器。
英文摘要:
An efficient policy search algorithm should estimate the local gradient of the objective function, with respect to the policy parameters, from as few trials as possible. Whereas most policy search methods estimate this gradient by observing the rewards obtained during policy trials, we show, both theoretically and empirically, that taking into account the sensor data as well gives better gradient estimates and hence faster learning. The reason is that rewards obtained during policy execution vary from trial to trial due to noise in the environment; sensor data, which correlates with the noise, can be used to partially correct for this variation, resulting in an estimatorwith lower variance.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。