arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1701.00867cs.AI
收藏

策略梯度算法中基线估计的K重方法

A K-fold Method for Baseline Estimation in Policy Gradient Algorithms

Samsung SDS Research America(三星 SDS 美国研究院);OpenAI;UC Berkeley(加州大学伯克利分校)

Nithyanand Kota, Abhishek Mishra, Sunil Srinivasa, Xi, Chen, Pieter Abbeel

摘要:

无偏策略梯度方法(如VPG和REINFORCE)中的高方差问题通常通过添加基线来缓解。然而,基线拟合本身遭受欠拟合或过拟合问题。在本文中,我们开发了一个K倍的基线估计方法在策略梯度算法。参数K是基线估计超参数,可以调整基线估计中的偏差-方差权衡。我们证明了我们的方法的有用性,通过两个国家的最先进的策略梯度算法的三个MuJoCo机车控制任务。

英文摘要:

The high variance issue in unbiased policy-gradient methods such as VPG and REINFORCE is typically mitigated by adding a baseline. However, the baseline fitting itself suffers from the underfitting or the overfitting problem. In this paper, we develop a K-fold method for baseline estimation in policy gradient algorithms. The parameter K is the baseline estimation hyperparameter that can adjust the bias-variance trade-off in the baseline estimates. We demonstrate the usefulness of our approach via two state-of-the-art policy gradient algorithms on three MuJoCo locomotive control tasks.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑