arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2209.09006cs.ROcs.LG
收藏

在机器人精确控制中加强轨迹优化和策略学习之间的一致性

Enforcing the consensus between Trajectory Optimization and Policy Learning for precise robot control

Inria（法国国家信息与自动化研究所）；Département d’Informatique de l’École normale supérieure, PSL Research University（巴黎文理研究大学巴黎高等师范学院计算机系）；LAAS-CNRS（法国国家科学研究中心系统分析与架构实验室）

Quentin Le Lidec, Wilson Jallet, Ivan Laptev, Cordelia Schmid, Justin Carpentier

阅读 PDF
arXiv 页面 ↗HTML 原文 ↗

摘要：

强化学习（RL）和轨迹优化（TO）具有很强的互补优势。一方面，强化学习方法能够直接从数据中学习全局控制策略，但通常需要大的样本量才能正确地收敛到可行的策略。另一方面，TO方法能够利用从模拟器中提取的基于梯度的信息快速收敛到仅在解附近有效的局部最优控制轨迹。在过去十年中，有几种方法旨在充分结合这两类方法，以获得两全其美。从这条线的研究之后，我们提出了一些改进，这些方法的顶部，更快地学习全局控制策略，特别是通过利用敏感性信息来自TO方法通过Sobolev学习，和增强拉格朗日技术，以加强TO和策略学习之间的共识。我们通过与文献中现有方法的比较，评估了这些改进对机器人技术中各种经典任务的好处。

英文摘要：

Reinforcement learning (RL) and trajectory optimization (TO) present strong complementary advantages. On one hand, RL approaches are able to learn global control policies directly from data, but generally require large sample sizes to properly converge towards feasible policies. On the other hand, TO methods are able to exploit gradient-based information extracted from simulators to quickly converge towards a locally optimal control trajectory which is only valid within the vicinity of the solution. Over the past decade, several approaches have aimed to adequately combine the two classes of methods in order to obtain the best of both worlds. Following on from this line of research, we propose several improvements on top of these approaches to learn global control policies quicker, notably by leveraging sensitivity information stemming from TO methods via Sobolev learning, and augmented Lagrangian techniques to enforce the consensus between TO and policy learning. We evaluate the benefits of these improvements on various classical tasks in robotics through comparison with existing approaches in the literature.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

↑