arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1910.03003cs.LGcs.ROcs.SYeess.SYstat.ML
收藏

作为近似输入推断的随机最优控制

Stochastic Optimal Control as Approximate Input Inference

Technische Universität Darmstadt(达姆施塔特工业大学);Max Planck Institute for Intelligent Systems(马克斯·普朗克智能系统研究所)

Joe Watson, Hany Abdulsamad, Jan Peters

摘要:

随机非线性动力系统的最优控制是机器人学习领域的一个主要挑战。鉴于全局控制问题的棘手性,最先进的算法集中在近似序列优化技术上,该技术严重依赖于正则化的遗传算法以实现稳定收敛。通过建立在推理和控制之间的对偶性,我们开发了输入估计的最优控制的观点,设计了一个概率随机最优控制公式,通过最小化控制成本的上界来迭代地推断最优输入分布。通过期望最大化和消息传递的动力系统的概率图模型进行推理,和时变线性高斯反馈控制器提取的联合状态-动作分布。这种观点结合了不确定性量化,通过先验的有效初始化,以及贝叶斯处理所固有的原则性正则化。此外,它可以表明,对于确定性线性化系统,我们的框架推导出最大熵线性二次型最优控制法律。我们提供了一个完整和详细的推导我们的概率方法,并强调其优势相比,其他确定性和概率求解器。

英文摘要:

Optimal control of stochastic nonlinear dynamical systems is a major challenge in the domain of robot learning. Given the intractability of the global control problem, state-of-the-art algorithms focus on approximate sequential optimization techniques, that heavily rely on heuristics for regularization in order to achieve stable convergence. By building upon the duality between inference and control, we develop the view of Optimal Control as Input Estimation, devising a probabilistic stochastic optimal control formulation that iteratively infers the optimal input distributions by minimizing an upper bound of the control cost. Inference is performed through Expectation Maximization and message passing on a probabilistic graphical model of the dynamical system, and time-varying linear Gaussian feedback controllers are extracted from the joint state-action distribution. This perspective incorporates uncertainty quantification, effective initialization through priors, and the principled regularization inherent to the Bayesian treatment. Moreover, it can be shown that for deterministic linearized systems, our framework derives the maximum entropy linear quadratic optimal control law. We provide a complete and detailed derivation of our probabilistic approach and highlight its advantages in comparison to other deterministic and probabilistic solvers.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑