Value Summation:一种用于基于MPC的模型强化学习的新型评分函数
Value Summation: A Novel Scoring Function for MPC-based Model-based Reinforcement Learning
AI总结:
针对基于MPC的强化学习规划模块用奖励函数评分轨迹存在固有偏差的问题,提出基于值折扣和的新型评分函数,结合最优轨迹引导与多源数据更新值函数,在多个基准测试中实现了优于现有SOTA算法的学习效率与奖励表现。
摘要:
本文针对基于MPC的强化学习方法规划模块中使用奖励函数为轨迹评分存在固有偏差的问题,提出了一种新型评分函数。所提方法利用值的折扣和提升了现有基于MPC的MBRL(基于模型的强化学习)方法的学习效率,该方法利用最优轨迹引导策略学习,并基于真实数据和增强的机载数据更新其状态-动作值函数。研究在选定的MuJoCo Gym环境以及Cassie机器人仿真模型的运动技能学习任务中评估了所提方法的学习效率。结果表明,该方法在学习效率和平均奖励回报方面均优于当前最先进的算法。
英文摘要:
This paper proposes a novel scoring function for the planning module of MPC-based reinforcement learning methods to address the inherent bias of using the reward function to score trajectories. The proposed method enhances the learning efficiency of existing MPC-based MBRL methods using the discounted sum of values. The method utilizes optimal trajectories to guide policy learning and updates its state-action value function based on real-world and augmented onboard data. The learning efficiency of the proposed method is evaluated in selected MuJoCo Gym environments as well as in learning locomotion skills for a simulated model of the Cassie robot. The results demonstrate that the proposed method outperforms the current state-of-the-art algorithms in terms of learning efficiency and average reward return.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。