基于类Lyapunov代理模型的观测学习镇定控制
Learning Stabilization Control from Observations by Learning Lyapunov-like Proxy Models
摘要：
将强化学习应用于机器人领域面临着奖励工程的困难。因此，方法集中于通过从观察中学习（LfO）来创建奖励函数，这是从仅包含状态序列的专家轨迹中学习策略的任务。我们提出了新的方法LfO的学习稳定的连续控制问题的重要类，通过引入中间代理模型作为奖励函数之间的专家和代理政策的基础上李雅普诺夫稳定性理论。我们的LfO培训过程包括两个步骤。第一步尝试从专家状态序列中学习一个类似于Lyapunov的景观代理模型，而不需要访问任何运动学模型，第二步使用学习的景观模型来指导学习者的策略。我们为这两个步骤制定了新的学习目标，这对整体培训的成功至关重要。我们评估我们的方法在真实的汽车机器人环境和其他模拟的稳定控制问题，在无模型设置，如四旋翼控制和保持直立位置的料斗在MuJoCo。我们与最先进的方法进行了比较，并表明所提出的方法可以有效地学习较少的专家意见。
英文摘要：
The deployment of Reinforcement Learning to robotics applications faces the difficulty of reward engineering. Therefore, approaches have focused on creating reward functions by Learning from Observations (LfO) which is the task of learning policies from expert trajectories that only contain state sequences. We propose new methods for LfO for the important class of continuous control problems of learning to stabilize, by introducing intermediate proxy models acting as reward functions between the expert and the agent policy based on Lyapunov stability theory. Our LfO training process consists of two steps. The first step attempts to learn a Lyapunov-like landscape proxy model from expert state sequences without access to any kinematics model, and the second step uses the learned landscape model to guide in training the learner's policy. We formulate novel learning objectives for the two steps that are important for overall training success. We evaluate our methods in real automobile robot environments and other simulated stabilization control problems in model-free settings, like Quadrotor control and maintaining upright positions of Hopper in MuJoCo. We compare with state-of-the-art approaches and show the proposed methods can learn efficiently with less expert observations.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
