Reinforcement learning has been applied to human movement through physiologically-based biomechanical models to add insights into the neural control of these movements; it is also useful in the design of prosthetics and robotics. In this paper, we extend the use of reinforcement learning into controlling an ocular biomechanical system to perform saccades, which is one of the fastest eye movement systems. We describe an ocular environment and an agent trained using Deep Deterministic Policy Gradients method to perform saccades. The agent was able to match the desired eye position with a mean deviation angle of 3:5+/-1:25 degrees. The proposed framework is a first step towards using the capabilities of deep reinforcement learning to enhance our understanding of ocular biomechanics.
MyoInteract: A Framework for Fast Prototyping of Biomechanical HCI Tasks using Reinforcement Learning
University of Cambridge（剑桥大学）；Center for Scalable Data Analytics and Artificial Intelligence (ScaDS.AI) Dresden/Leipzig, Leipzig University（可扩展数据与人工智能分析中心（ScaDS.AI）德累斯顿/莱比锡，莱比锡大学）
Motor control is a set of time-varying muscle excitations which generate desired motions for a biomechanical system. Muscle excitations cannot be directly measured from live subjects. An alternative approach is to estimate muscle activations using inverse motion-driven simulation. In this article, we propose a deep reinforcement learning method to estimate the muscle excitations in simulated biomechanical systems. Here, we introduce a custom-made reward function which incentivizes faster point-to-point tracking of target motion. Moreover, we deploy two new techniques, namely, episode-based har…
Among the infinite number of possible movements that can be produced, humans are commonly assumed to choose those that optimize criteria such as minimizing movement time, subject to certain movement constraints like signal-dependent and constant motor noise. While so far these assumptions have only been evaluated for simplified point-mass or planar models, we address the question of whether they can predict reaching movements in a full skeletal model of the human upper extremity. We learn a control policy using a motor babbling approach as implemented in reinforcement learning, using aimed mov…
It is not until we become senior citizens do we recognise how much we took maintaining a simple standing posture for granted. It is truly fascinating to observe the magnitude of control the human brain exercises, in real time, to activate and deactivate the lower body muscles and solve a multi-link 3D inverted pendulum problem in order to maintain a stable standing posture. This realisation is even more apparent when training an artificial intelligence (AI) agent to maintain a standing posture of a digital musculoskeletal avatar due to the error propagation problem. In this work we address the…
To diagnose, plan, and treat musculoskeletal pathologies, understanding and reproducing muscle recruitment for complex movements is essential. With muscle activations for movements often being highly redundant, nonlinear, and time dependent, machine learning can provide a solution for their modeling and control for anatomy-specific musculoskeletal simulations. Sophisticated biomechanical simulations often require specialized computational environments, being numerically complex and slow, hindering their integration with typical deep learning frameworks. In this work, a deep reinforcement learn…
Log2Motion: Biomechanical Motion Synthesis from Touch Logs
ScaDS.AI, Leipzig University Leipzig Germany；Aalto University \& ELLIS Institute Finland Helsinki Finland；ScaDS.AI, Leipzig University；Aalto University \& ELLIS Institute Finland
Synthesizing physiologically-accurate human movement in a variety of conditions can help practitioners plan surgeries, design experiments, or prototype assistive devices in simulated environments, reducing time and costs and improving treatment outcomes. Because of the large and complex solution spaces of biomechanical models, current methods are constrained to specific movements and models, requiring careful design of a controller and hindering many possible applications. We sought to discover if modern optimization methods efficiently explore these complex spaces. To do this, we posed the pr…
Department of Mechanical Engineering（机械工程系）；Carnegie Mellon University（卡内基梅隆大学）；Department of Physical Education（体育系）；Seoul National University（首尔国立大学）；Lampe Joint Department of Biomedical Engineering（生物医学工程联合部门）；UNC-Chapel Hill and NC State University（北卡罗来纳大学教堂山分校和北卡罗来纳州立大学）
Natural and Robust Walking using Reinforcement Learning without Demonstrations in High-Dimensional Musculoskeletal Models
Max-Planck Institute for Intelligent Systems（马克斯·普朗克智能系统研究所）；Hertie Institute for Clinical Brain Research（赫蒂临床脑研究所）；Center for Integrative Neuroscience（综合神经科学中心）；Goatstream；Meta AI；University of Stuttgart（斯图加特大学）；University of Tübingen（图宾根大学）；Heidelberg University（海德堡大学）
Attention Trajectories as a Diagnostic Axis for Deep Reinforcement Learning
Max Planck Institute for Human Cognitive and Brain Sciences（马克斯·普朗克人类认知与脑科学研究所）；Center for Scalable Data Analytics and Artificial Intelligence (ScaDS.AI)（可扩展数据与人工智能研究中心）；Leipzig University（莱比锡大学）
RLPeri: Accelerating Visual Perimetry Test with Reinforcement Learning and Convolutional Feature Extraction
Institute of High Performance Computing (IHPC), Agency for Science, Technology and Research (A*STAR)（新加坡科技研究局高性能计算研究院）；Singapore Eye Research Institute, Singapore National Eye Centre（新加坡眼科研究所，新加坡全国眼科中心）
Deakin University（迪肯大学）；Medical Research Institute, Alexandria University（亚历山大大学医学研究所）；Alexandria University（亚历山大大学）；University of Technology Sydney（悉尼科技大学）
In this paper, we propose enhancing actor-critic reinforcement learning agents by parameterising the final actor layer which produces the actions in order to accommodate the behaviour discrepancy of different actuators, under different load conditions during interaction with the environment. We propose branching the action producing layer in the actor to learn the tuning parameter controlling the activation layer (e.g. Tanh and Sigmoid). The learned parameters are then used to create tailored activation functions for each actuator. We ran experiments on three OpenAI Gym environments, i.e. Pend…
University of Khartoum（喀土穆大学）；University of Witwatersrand（金山大学）；CSIR（南非科学与工业研究理事会）
In this paper, We Apply Reinforcement learning (RL) techniques to train a realistic biomechanical model to work with different people and on different walking environments. We benchmarking 3 RL algorithms: Deep Deterministic Policy Gradient (DDPG), Trust Region Policy Optimization (TRPO) and Proximal Policy Optimization (PPO) in OpenSim environment, Also we apply imitation learning to a prosthetics domain to reduce the training time needed to design customized prosthetics. We use DDPG algorithm to train an original expert agent. We then propose a modification to the Dataset Aggregation (DAgger…
Real-time autonomous magnetic microrobot navigation across dynamic and biologically relevant environments
Department of Mechanical Engineering, University of Delaware（德克萨斯大学机械工程系）；Department of Animal and Food Sciences, University of Delaware（德克萨斯大学动物与食品科学系）；School of Civil and Environmental Engineering, Cornell University（康奈尔大学土木与环境工程学院）
