arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2212.02587cs.ROcs.AIcs.SYeess.SY
收藏

模型预测控制的采样分布学习

Learning Sampling Distributions for Model Predictive Control

University of Washington(华盛顿大学)

Jacob Sacks, Byron Boots

摘要:

基于采样的方法已经成为当代模型预测控制(MPC)方法的基石,因为它们对动态或成本函数的可微性没有限制,并且易于并行化。然而,它们的功效高度依赖于采样分布本身的质量,这通常被假设为简单的,如高斯分布。这种限制可能会导致样本远非最佳,从而导致性能低下。最近的工作已经探索通过在控制的学习潜在空间中采样来改善MPC的性能。然而,这些方法最终在控制空间中的时间步长之间执行所有MPC参数更新和热启动。这要求我们依赖于许多算法来生成样本和更新分布,并可能导致次优性能。相反,我们建议在潜在空间中执行所有操作,使我们能够充分利用学习的分布。具体来说,我们框架的学习问题作为两个层次的优化,并显示如何训练控制器与反向传播通过时间。通过使用分布的归一化流参数化,我们可以利用其易于处理的密度来避免要求动态和成本函数的可微性。最后,我们在模拟机器人任务上评估了所提出的方法,并证明了它能够超越现有方法的性能,并在减少样本数量的情况下更好地扩展。

英文摘要:

Sampling-based methods have become a cornerstone of contemporary approaches to Model Predictive Control (MPC), as they make no restrictions on the differentiability of the dynamics or cost function and are straightforward to parallelize. However, their efficacy is highly dependent on the quality of the sampling distribution itself, which is often assumed to be simple, like a Gaussian. This restriction can result in samples which are far from optimal, leading to poor performance. Recent work has explored improving the performance of MPC by sampling in a learned latent space of controls. However, these methods ultimately perform all MPC parameter updates and warm-starting between time steps in the control space. This requires us to rely on a number of heuristics for generating samples and updating the distribution and may lead to sub-optimal performance. Instead, we propose to carry out all operations in the latent space, allowing us to take full advantage of the learned distribution. Specifically, we frame the learning problem as bi-level optimization and show how to train the controller with backpropagation-through-time. By using a normalizing flow parameterization of the distribution, we can leverage its tractable density to avoid requiring differentiability of the dynamics and cost function. Finally, we evaluate the proposed approach on simulated robotics tasks and demonstrate its ability to surpass the performance of prior methods and scale better with a reduced number of samples.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑