Deep reinforcement learning (DRL) has been successfully used to solve various robotic manipulation tasks. However, most of the existing works do not address the issue of control stability. This is in sharp contrast to the control theory community where the well-established norm is to prove stability whenever a control law is synthesized. What makes traditional stability analysis difficult for DRL are the uninterpretable nature of the neural network policies and unknown system dynamics. In this work, stability is obtained by deriving an interpretable deep policy structure based on the $\textit{energy shaping}$ control of Lagrangian systems. Then, stability during physical interaction with an unknown environment is established based on $\textit{passivity}$. The result is a stability guaranteeing DRL in a model-free framework that is general enough for contact-rich manipulation tasks. With an experiment on a peg-in-hole task, we demonstrate, to the best of our knowledge, the first DRL with stability guarantee on a real robotic manipulator.
SSP: Safety-guaranteed Surgical Policy via Joint Optimization of Behavioral and Spatial Constraints
Global College, Shanghai Jiao Tong University（上海交通大学全球学院）；Department of Computer Science and Engineering, The Chinese University of Hong Kong（香港中文大学计算机科学与工程系）；Computer Science and Artificial Intelligence Laboratory, Massachusetts Institute of Technology（麻省理工学院计算机科学与人工智能实验室）
Interactive Shaping of Granular Media Using Reinforcement Learning
Humanoid Robots Lab, University of Bonn, Germany（波恩大学）；Autonomous Intelligent Systems Lab, University of Bonn, Germany（波恩大学）；Center for Robotics, University of Bonn, Germany（波恩大学）；Lamarr Institute for ML and AI, Bonn, Germany（波恩人工智能与机器学习研究所）
Tactical Reward Shaping: Bypassing Reinforcement Learning with Strategy-Based Goals
ShanghaiTech University（上海科技大学）
深度强化学习（DRL）已经显示出其有前途的能力，可以直接从试错中学习最优策略。然而，如果奖励函数定义的学习目标“不是最佳”，学习可能会受到阻碍。我们证明，通过以一种反直觉但智能的方式设置竞争的目标/目标，而不是通过许多小时的DRL模拟来尝试解决方案，可以快速收敛到一个获胜的策略。ICRA-DJI RoboMaster AI Challenge是一款在部分可观察环境中进行机器人之间合作和竞争的游戏，与Counter-Strike游戏非常相似。与传统的游戏方法不同，奖励是在赢得比赛或击中敌人时给予的，我们的DRL算法在处于几何战略优势时奖励我们的机器人，这隐含地增加了获胜的机会。此外，我们使用深度Q学习（DQL）来生成移动的多智能体路径，通过避免碰撞来改善两个机器人之间的合作。最后，我们实现了一个变种A* 算法与DQL相同的隐式几何目标和比较结果。我们的结论是，一个良好的设定的目标可以提出问题的需要学习算法，与几何为基础的搜索优于DQL在许多数量级。
