Deep brain stimulation (DBS) has shown great promise toward treating motor symptoms caused by Parkinson's disease (PD), by delivering electrical pulses to the Basal Ganglia (BG) region of the brain. However, DBS devices approved by the U.S. Food and Drug Administration (FDA) can only deliver continuous DBS (cDBS) stimuli at a fixed amplitude; this energy inefficient operation reduces battery lifetime of the device, cannot adapt treatment dynamically for activity, and may cause significant side-effects (e.g., gait impairment). In this work, we introduce an offline reinforcement learning (RL) framework, allowing the use of past clinical data to train an RL policy to adjust the stimulation amplitude in real time, with the goal of reducing energy use while maintaining the same level of treatment (i.e., control) efficacy as cDBS. Moreover, clinical protocols require the safety and performance of such RL controllers to be demonstrated ahead of deployments in patients. Thus, we also introduce an offline policy evaluation (OPE) method to estimate the performance of RL policies using historical data, before deploying them on patients. We evaluated our framework on four PD patients equipped with the RC+S DBS system, employing the RL controllers during monthly clinical visits, with the overall control efficacy evaluated by severity of symptoms (i.e., bradykinesia and tremor), changes in PD biomakers (i.e., local field potentials), and patient ratings. The results from clinical experiments show that our RL-based controller maintains the same level of control efficacy as cDBS, but with significantly reduced stimulation energy. Further, the OPE method is shown effective in accurately estimating and ranking the expected returns of RL controllers.
Sample-Efficient Reinforcement Learning Controller for Deep Brain Stimulation in Parkinson's Disease
Department of Electrical and Computer Engineering, Clemson University（电子与计算机工程系，克莱姆森大学）；Department of Biomedical Engineering, Michigan Technological University（生物医学工程系，密歇根技术大学）
Skolkovo Institute of Science and Technology（斯科尔科沃科学技术研究所）；Microsoft Research Lab（微软研究院实验室）；University of Potsdam（波茨坦大学）
Malfunctioning neurons in the brain sometimes operate synchronously, reportedly causing many neurological diseases, e.g. Parkinson's. Suppression and control of this collective synchronous activity are therefore of great importance for neuroscience, and can only rely on limited engineering trials due to the need to experiment with live human brains. We present the first Reinforcement Learning gym framework that emulates this collective behavior of neurons and allows us to find suppression parameters for the environment of synthetic degenerate models of neurons. We successfully suppress synchro…
Department of Computer Science, UNC Chapel Hill（UNC夏洛茨维尔大学计算机科学系）；Hochschule München（慕尼黑应用科学大学）；Department of Neurosurgery, UNC Chapel Hill（UNC夏洛茨维尔大学神经外科系）；TU Munich Institute for Advanced Study（慕尼黑技术大学高级研究学院）
Neuromorphic Energy-Aware Learning for Adaptive Deep Brain Stimulation
Dept. of Electrical and Computer Engineering, University of California, Santa Cruz, CA, USA（加州大学圣克鲁兹分校电气与计算机工程系）；Dept. of Bioengineering, University of California, San Diego, La Jolla, CA, USA（加州大学圣地亚哥分校生物工程系）
Deep learning for spike detection in deep brain stimulation surgery
Department of Electronics and Information Technology, Warsaw University of Technology（华沙理工大学电子与信息技术系）；Research and Academic Computer Network (NASK-PIB)（研究与学术计算机网络（NASK-PIB））
Solarillion Foundation（Solarillion基金会）；SRM Institute of Science and Technology（SRM科学技术学院）
Freezing of Gait (FoG) is a common gait deficit among patients diagnosed with Parkinson's Disease (PD). In order to help these patients recover from FoG episodes, Rhythmic Auditory Stimulation (RAS) is needed. The authors propose a ubiquitous embedded system that detects FOG events with a Machine Learning (ML) subsystem from accelerometer signals . By making inferences on-device, we avoid issues prevalent in cloud-based systems such as latency and network connection dependency. The resource-efficient classifier used, reduces the model size requirements by approximately 400 times compared to th…
London School of Economics and Political Science（伦敦经济政治学院）；University of California at Berkeley（加州大学伯克利分校）；The Alan Turing Institute（艾伦·图灵研究所）；University of Manchester（曼彻斯特大学）
Robust Neural Stimulation Response Modeling Through Meta-Learning and Pretraining
University of Washington（华盛顿大学）；Paul G. Allen School of Computer Science & Engineering（保罗·G·艾伦计算机科学与工程学院）；Center for Neurotechnology（神经技术中心）；Computational Neuroscience Center（计算神经科学中心）；Department of Electrical and Computer Engineering（电气与计算机工程系）；Department of Bioengineering（生物工程系）
The University of Texas at Dallas（德克萨斯大学达拉斯分校）；The University of Texas at Austin（德克萨斯大学奥斯汀分校）
Fundamental knowledge in activity recognition of individuals with motor disorders such as Parkinson's disease (PD) has been primarily limited to detection of steady-state/static tasks (sitting, standing, walking). To date, identification of non-steady-state locomotion on uneven terrains (stairs, ramps) has not received much attention. Furthermore, previous research has mainly relied on data from a large number of body locations which could adversely affect user convenience and system performance. Here, individuals with mild stages of PD and healthy subjects performed non-steady-state circuit t…
Temporal Basis Function Models for Closed-Loop Neural Stimulation
1 Neural Systems Laboratory, Paul G. Allen School of Computer Science \& Engineering, University of Washington, Seattle, WA, USA；2 Center for Neurotechnology, University of Washington, Seattle, WA, USA；3 Computational Neuroscience Center, University of Washington, Seattle, WA, USA；4 Department of Bioengineering, University of Washington, Seattle, WA, USA；5 Department of Electrical；Computer Engineering, University of Washington, Seattle, WA, USA
Microelectrode Signal Dynamics as Biomarkers of Subthalamic Nucleus Entry on Deep Brain Stimulation: A Nonlinear Feature Approach
Institute of Biological Sciences（生物科学研究所）；Federal University of Pará（帕拉联邦大学）；Hospital Ophir Loyola（奥菲尔洛伊拉医院）；Faculty of Medical Sciences（医学科学学院）；UNICAMP（UNICAMP大学）；Cell Biology Graduate Program（细胞生物学研究生项目）
A swap-adversarial framework for improving domain generalization in electrocorticography-based Parkinson's disease classification
Department of Computer Science and Engineering, Incheon National University（仁川国立大学计算机科学与工程系）；Department of Nanobioengineering, Incheon National University（仁川国立大学纳米生物工程系）；Center for Brain-Machine Interface, Incheon National University（仁川国立大学脑机接口中心）；Department of Artificial Intelligence, Inha University（仁荷大学人工智能系）
The detection of human sleep stages is widely used in the diagnosis and intervention of neurological and psychiatric diseases. Some patients with deep brain stimulator implanted could have their neural activities recorded from the deep brain. Sleep stage classification based on deep brain recording has great potential to provide more precise treatment for patients. The accuracy and generalizability of existing sleep stage classifiers based on local field potentials are still limited. We proposed an applicable cross-modal transfer learning method for sleep stage classification with implanted de…
University of Freiburg（弗赖堡大学）；Radboud University（拉德堡德大学）
The decoding of brain signals recorded via, e.g., an electroencephalogram, using machine learning is key to brain-computer interfaces (BCIs). Stimulation parameters or other experimental settings of the BCI protocol typically are chosen according to the literature. The decoding performance directly depends on the choice of parameters, as they influence the elicited brain signals and optimal parameters are subject-dependent. Thus a fast and automated selection procedure for experimental parameters could greatly improve the usability of BCIs. We evaluate a standalone random search and a combined…
Insititute for Robots and Intelligent Machines（机器人与智能机器研究所）；Georgia Institute of Technology（佐治亚理工学院）；Emory University（埃默里大学）
Learning to control a safety-critical system with latent dynamics (e.g. for deep brain stimulation) requires taking calculated risks to gain information as efficiently as possible. To address this problem, we present a probabilistically-safe, meta-active learning approach to efficiently learn system dynamics and optimal configurations. We cast this problem as meta-learning an acquisition function, which is represented by a Long-Short Term Memory Network (LSTM) encoding sampling history. This acquisition function is meta-learned offline to learn high quality sampling strategies. We employ a mix…
