基于Actor-Critic强化学习的机器人膝关节跟踪控制
Robotic Knee Tracking Control to Mimic the Intact Human Knee Profile Based on Actor-critic Reinforcement Learning
摘要：
我们解决了一个国家的最先进的强化学习（RL）的控制方法，自动配置机器人假体阻抗参数，使端到端，连续的运动，旨在为经股截肢受试者。具体来说，我们的演员-评论家基于RL提供跟踪控制的机器人膝关节假体模仿完整的膝关节轮廓。这是一个显着的进步，从我们以前的RL为基础的自动调整假肢控制参数，集中在调节控制与设计师规定的机器人膝关节轮廓作为目标。除了提出完整的跟踪控制算法的基础上直接启发式动态规划（dHDP），我们提供了一个分析框架的跟踪控制器的约束输入。我们证明了我们提出的跟踪控制具有几个重要的性质，如学习网络的权值收敛，Bellman（次）最优的成本去价值函数和控制输入，和实际稳定的人-机器人系统下的输入约束。我们还提供了一个系统的模拟所提出的跟踪控制使用一个现实的人类-机器人系统模拟器，OpenSim，模拟如何dHDP使平地行走，行走在不同的地形和不同的步伐。这些结果表明，我们提出的基于dHDP的跟踪控制不仅在理论上是合适的，而且在实践中也是有用的。
英文摘要：
We address a state-of-the-art reinforcement learning (RL) control approach to automatically configure robotic prosthesis impedance parameters to enable end-to-end, continuous locomotion intended for transfemoral amputee subjects. Specifically, our actor-critic based RL provides tracking control of a robotic knee prosthesis to mimic the intact knee profile. This is a significant advance from our previous RL based automatic tuning of prosthesis control parameters which have centered on regulation control with a designer prescribed robotic knee profile as the target. In addition to presenting the complete tracking control algorithm based on direct heuristic dynamic programming (dHDP), we provide an analytical framework for the tracking controller with constrained inputs. We show that our proposed tracking control possesses several important properties, such as weight convergence of the learning networks, Bellman (sub)optimality of the cost-to-go value function and control input, and practical stability of the human-robot system under input constraint. We further provide a systematic simulation of the proposed tracking control using a realistic human-robot system simulator, the OpenSim, to emulate how the dHDP enables level ground walking, walking on different terrains and at different paces. These results show that our proposed dHDP based tracking control is not only theoretically suitable, but also practically useful.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
