arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1910.03358cs.LGcs.ROstat.ML
收藏

深值模型预测控制

Deep Value Model Predictive Control

ETH Zurich(苏黎世联邦理工学院);NVIDIA(英伟达)

Farbod Farshidian, David Hoeller, Marco Hutter

摘要:

在本文中,我们介绍了一个行动者-评论算法称为深度值模型预测控制(DMPC),它结合了基于模型的轨迹优化与价值函数估计。DMPC执行者是一个模型预测控制(MPC)优化器,其目标函数根据评价者估计的价值函数来定义。我们表明,我们的MPC演员是一个重要的采样器,最小化的最优采样策略的状态分布的交叉熵的上限。在我们与Ballbot系统的实验中,我们表明我们的算法可以使用稀疏和二进制奖励信号来有效地解决避障和目标到达任务。与以前的工作相比,我们表明,包括价值函数的运行成本的轨迹优化加快收敛。我们还讨论了必要的策略,鲁棒性的算法在实践中。

英文摘要:

In this paper, we introduce an actor-critic algorithm called Deep Value Model Predictive Control (DMPC), which combines model-based trajectory optimization with value function estimation. The DMPC actor is a Model Predictive Control (MPC) optimizer with an objective function defined in terms of a value function estimated by the critic. We show that our MPC actor is an importance sampler, which minimizes an upper bound of the cross-entropy to the state distribution of the optimal sampling policy. In our experiments with a Ballbot system, we show that our algorithm can work with sparse and binary reward signals to efficiently solve obstacle avoidance and target reaching tasks. Compared to previous work, we show that including the value function in the running cost of the trajectory optimizer speeds up the convergence. We also discuss the necessary strategies to robustify the algorithm in practice.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑