arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2108.06911cs.LGcs.SYeess.SY
收藏

基于优化训练数据集的最优Actor-Critic策略

Optimal Actor-Critic Policy with Optimized Training Datasets

University of Newcastle(纽卡斯尔大学);Shiraz University of Medical Sciences(设拉子医科大学)

Chayan Banerjee, Zhiyong Chen, Nasimul Noman, Mohsen Zamani

摘要:

Actor-Critic(AC)算法以其在解决强化学习问题中的有效性和高性能而闻名,但它们也遭受低采样效率。基于AC的策略优化过程是迭代的,并且需要频繁地访问代理-环境系统以通过推出策略、收集奖励和状态(即样本)以及从它们学习来评估和更新策略。它最终需要大量的样本来学习最优策略。为了提高采样效率,我们提出了一种策略来优化训练数据集,其中包含从AC过程中收集的显着较少的样本。数据集优化是由一个最佳的情节只有操作,一个政策参数适应度模型,和一个遗传算法模块。与许多当代AC算法相比,由优化的训练数据集训练的最优策略网络在控制自治动态系统方面表现出优越的性能。标准基准测试结果表明,该方法提高了采样效率,确保更快地收敛到最优值,并具有更高的数据效率。

英文摘要:

Actor-critic (AC) algorithms are known for their efficacy and high performance in solving reinforcement learning problems, but they also suffer from low sampling efficiency. An AC based policy optimization process is iterative and needs to frequently access the agent-environment system to evaluate and update the policy by rolling out the policy, collecting rewards and states (i.e. samples), and learning from them. It ultimately requires a huge number of samples to learn an optimal policy. To improve sampling efficiency, we propose a strategy to optimize the training dataset that contains significantly less samples collected from the AC process. The dataset optimization is made of a best episode only operation, a policy parameter-fitness model, and a genetic algorithm module. The optimal policy network trained by the optimized training dataset exhibits superior performance compared to many contemporary AC algorithms in controlling autonomous dynamical systems. Evaluation on standard benchmarks show that the method improves sampling efficiency, ensures faster convergence to optima, and is more data-efficient than its counterparts.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑