Department of Mechanical Engineering（机械工程系）；Carnegie Mellon University（卡内基梅隆大学）；Department of Physical Education（体育系）；Seoul National University（首尔国立大学）；Lampe Joint Department of Biomedical Engineering（生物医学工程联合部门）；UNC-Chapel Hill and NC State University（北卡罗来纳大学教堂山分校和北卡罗来纳州立大学）
Ilseung Park, Eunsik Choi, Jangwhan Ahn, Jooeun Ahn
Human locomotion emerges from high-dimensional neuromuscular control, making predictive musculoskeletal simulation challenging. We present a physiology-informed reinforcement-learning framework that constrains control using muscle synergies. We extracted a low-dimensional synergy basis from inverse musculoskeletal analyses of a small set of overground walking trials and used it as the action space for a muscle-driven three-dimensional model trained across variable speeds, slopes and uneven terrain. The resulting controller generated stable gait from 0.7-1.8 m/s and on $\pm$ 6$^{\circ}$ grades and reproduced condition-dependent modulation of joint angles, joint moments and ground reaction forces. Compared with an unconstrained controller, synergy-constrained control reduced non-physiological knee kinematics and kept knee moment profiles within the experimental envelope. Across conditions, simulated vertical ground reaction forces correlated strongly with human measurements, and muscle-activation timing largely fell within inter-subject variability. These results show that embedding neurophysiological structure into reinforcement learning can improve biomechanical fidelity and generalization in predictive human locomotion simulation with limited experimental data.
Motor control is a set of time-varying muscle excitations which generate desired motions for a biomechanical system. Muscle excitations cannot be directly measured from live subjects. An alternative approach is to estimate muscle activations using inverse motion-driven simulation. In this article, we propose a deep reinforcement learning method to estimate the muscle excitations in simulated biomechanical systems. Here, we introduce a custom-made reward function which incentivizes faster point-to-point tracking of target motion. Moreover, we deploy two new techniques, namely, episode-based har…
Royal Veterinary College（皇家兽医学院）；DeepMind（深度思维）；University of California, Irvine（加州大学欧文分校）；Imperial College London（帝国理工学院）
Muscle-actuated control is a research topic that spans multiple domains, including biomechanics, neuroscience, reinforcement learning, robotics, and graphics. This type of control is particularly challenging as bodies are often overactuated and dynamics are delayed and non-linear. It is however a very well tested and tuned actuation mechanism that has undergone millions of years of evolution with interesting properties exploiting passive forces and efficient energy storage of muscle-tendon units. To facilitate research on muscle-actuated simulation, we release a 3D musculoskeletal simulation o…
To diagnose, plan, and treat musculoskeletal pathologies, understanding and reproducing muscle recruitment for complex movements is essential. With muscle activations for movements often being highly redundant, nonlinear, and time dependent, machine learning can provide a solution for their modeling and control for anatomy-specific musculoskeletal simulations. Sophisticated biomechanical simulations often require specialized computational environments, being numerically complex and slow, hindering their integration with typical deep learning frameworks. In this work, a deep reinforcement learn…
DEP-RL: Embodied Exploration for Reinforcement Learning in Overactuated and Musculoskeletal Systems
Max Planck Institute for Intelligent Systems（马克斯·普朗克智能系统研究所）；Hertie-Institute for Clinical Brain Research（赫蒂临床脑研究所）；Institute for Modelling and Simulation of Biomechanical Systems（生物力学系统建模与仿真研究所）
Natural and Robust Walking using Reinforcement Learning without Demonstrations in High-Dimensional Musculoskeletal Models
Max-Planck Institute for Intelligent Systems（马克斯·普朗克智能系统研究所）；Hertie Institute for Clinical Brain Research（赫蒂临床脑研究所）；Center for Integrative Neuroscience（综合神经科学中心）；Goatstream；Meta AI；University of Stuttgart（斯图加特大学）；University of Tübingen（图宾根大学）；Heidelberg University（海德堡大学）
Lambda-Hold Control: Human-Like Movement Emerges from a Minimal Task Reward in Predictive Musculoskeletal Simulation
Institute of Sport Science, Seoul National University（首尔国立大学体育科学研究所）；SNU Robotics Institute, Seoul National University（首尔国立大学SNU机器人研究所）；Department of Physical Education, Seoul National University（首尔国立大学体育学院）
Imitation Learning for Adaptive Control of a Virtual Soft Exoglove
Department of Informatics, King’s College London（信息学院，伦敦国王学院）；MyoLab；Harvard Medical School（哈佛医学院）；Spaulding Rehabilitation Hospital（斯波林康复医院）；Department of Bioengineering, Faculty of Engineering, Imperial College London（生物工程系，伦敦帝国理工学院工程学院）；John A. Paulson School of Engineering and Applied Sciences, Harvard University（哈佛大学约翰·A·保罗森工程与应用科学学院）
Among the infinite number of possible movements that can be produced, humans are commonly assumed to choose those that optimize criteria such as minimizing movement time, subject to certain movement constraints like signal-dependent and constant motor noise. While so far these assumptions have only been evaluated for simplified point-mass or planar models, we address the question of whether they can predict reaching movements in a full skeletal model of the human upper extremity. We learn a control policy using a motor babbling approach as implemented in reinforcement learning, using aimed mov…
Functional Electrical Stimulation (FES) can restore motion to a paralysed person's muscles. Yet, control stimulating many muscles to restore the practical function of entire limbs is an unsolved problem. Current neurostimulation engineering still relies on 20th Century control approaches and correspondingly shows only modest results that require daily tinkering to operate at all. Here, we present our state of the art Deep Reinforcement Learning (RL) developed for real time adaptive neurostimulation of paralysed legs for FES cycling. Core to our approach is the integration of a personalised neu…
Meta AI Research（Meta人工智能研究院）；University of Twente（特文特大学）
Embodied agents in continuous control domains have had limited exposure to tasks allowing to explore musculoskeletal properties that enable agile and nimble behaviors in biological beings. The sophistication behind neuro-musculoskeletal control can pose new challenges for the motor learning community. At the same time, agents solving complex neural control problems allow impact in fields such as neuro-rehabilitation, as well as collaborative-robotics. Human biomechanics underlies complex multi-joint-multi-actuator musculoskeletal systems. The sensory-motor system relies on a range of sensory-c…
We present Model-Predictive Interaction Primitives -- a robot learning framework for assistive motion in human-machine collaboration tasks which explicitly accounts for biomechanical impact on the human musculoskeletal system. First, we extend Interaction Primitives to enable predictive biomechanics: the prediction of future biomechanical states of a human partner conditioned on current observations and intended robot control signals. In turn, we leverage this capability within a model-predictive control strategy to identify the future ergonomic and biomechanical ramifications of potential rob…
PerSival: Neural-network-based visualisation for pervasive continuum-mechanical simulations in musculoskeletal biomechanics
Institute for Parallel and Distributed Systems, University of Stuttgart（并行与分布式系统研究所，斯图加特大学）；Visualization Research Center VISUS, University of Stuttgart（可视化研究中心VISUS，斯图加特大学）；Biomechatronic Systems, Fraunhofer IPA, Stuttgart（生物机械系统，弗劳恩霍夫IPA研究所，斯图加特）
Universiti Kebangsaan Malaysia（马来西亚国民大学）；Qatar University（卡塔尔大学）；Dr Ram Manohar Lohia Avadh University（拉姆·马诺哈尔·洛希亚阿瓦德大学）；Xiamen University Malaysia（厦门大学马来西亚分校）；National Research and Innovation Agency（国家研究与创新署）
Diabetic sensorimotor polyneuropathy (DSPN) is one of the prevalent forms of neuropathy affected by diabetic patients that involves alterations in biomechanical changes in human gait. In literature, for the last 50 years, researchers are trying to observe the biomechanical changes due to DSPN by studying muscle electromyography (EMG), and ground reaction forces (GRF). However, the literature is contradictory. In such a scenario, we are proposing to use Machine learning techniques to identify DSPN patients by using EMG, and GRF data. We have collected a dataset consists of three lower limb musc…
Ultrasound imaging provides information from a large part of the muscle. It has recently been shown that ultrafast ultrasound imaging can be used to record and analyze the mechanical response of individual MUs using blind source separation. In this work, we present an alternative method - a deep learning pipeline - to identify active MUs in ultrasound image sequences, including segmentation of their territories and signal estimation of their mechanical responses (twitch train). We train and evaluate the model using simulated data mimicking the complex activation pattern of tens of activated MU…
