Malfunctioning neurons in the brain sometimes operate synchronously, reportedly causing many neurological diseases, e.g. Parkinson's. Suppression and control of this collective synchronous activity are therefore of great importance for neuroscience, and can only rely on limited engineering trials due to the need to experiment with live human brains. We present the first Reinforcement Learning gym framework that emulates this collective behavior of neurons and allows us to find suppression parameters for the environment of synthetic degenerate models of neurons. We successfully suppress synchrony via RL for three pathological signaling regimes, characterize the framework's stability to noise, and further remove the unwanted oscillations by engaging multiple PPO agents.
Sample-Efficient Reinforcement Learning Controller for Deep Brain Stimulation in Parkinson's Disease
Department of Electrical and Computer Engineering, Clemson University（电子与计算机工程系，克莱姆森大学）；Department of Biomedical Engineering, Michigan Technological University（生物医学工程系，密歇根技术大学）
Department of Computer Science, UNC Chapel Hill（UNC夏洛茨维尔大学计算机科学系）；Hochschule München（慕尼黑应用科学大学）；Department of Neurosurgery, UNC Chapel Hill（UNC夏洛茨维尔大学神经外科系）；TU Munich Institute for Advanced Study（慕尼黑技术大学高级研究学院）
Neuromorphic Energy-Aware Learning for Adaptive Deep Brain Stimulation
Dept. of Electrical and Computer Engineering, University of California, Santa Cruz, CA, USA（加州大学圣克鲁兹分校电气与计算机工程系）；Dept. of Bioengineering, University of California, San Diego, La Jolla, CA, USA（加州大学圣地亚哥分校生物工程系）
Deep brain stimulation (DBS) has shown great promise toward treating motor symptoms caused by Parkinson's disease (PD), by delivering electrical pulses to the Basal Ganglia (BG) region of the brain. However, DBS devices approved by the U.S. Food and Drug Administration (FDA) can only deliver continuous DBS (cDBS) stimuli at a fixed amplitude; this energy inefficient operation reduces battery lifetime of the device, cannot adapt treatment dynamically for activity, and may cause significant side-effects (e.g., gait impairment). In this work, we introduce an offline reinforcement learning (RL) fr…
Human movement disorders or paralysis lead to the loss of control of muscle activation and thus motor control. Functional Electrical Stimulation (FES) is an established and safe technique for contracting muscles by stimulating the skin above a muscle to induce its contraction. However, an open challenge remains on how to restore motor abilities to human limbs through FES, as the problem of controlling the stimulation is unclear. We are taking a robotics perspective on this problem, by developing robot learning algorithms that control the ultimate humanoid robot, the human body, through electri…
Functional Electrical Stimulation (FES) is a technique to evoke muscle contraction through low-energy electrical signals. FES can animate paralysed limbs. Yet, an open challenge remains on how to apply FES to achieve desired movements. This challenge is accentuated by the complexities of human bodies and the non-stationarities of the muscles' responses. The former causes difficulties in performing inverse dynamics, and the latter causes control performance to degrade over extended periods of use. Here, we engage the challenge via a data-driven approach. Specifically, we learn to control FES th…
Stimulation-based control of dynamic brain networks
University of Pennsylvania（宾夕法尼亚大学）；US Army Research Laboratory（美国陆军研究实验室）；University at Buffalo, SUNY（纽约州立大学布法罗分校）；University of California, Riverside（加州大学河滨分校）；University of California, Santa Barbara（加州大学圣塔芭芭拉分校）
The brain stimulation and its widespread use is one of the most important subjects in studies of neurophysiology. In brain electrical stimulation methods, following the surgery and electrode implantation, electrodes send electrical impulses to the specific targets in the brain. The use of this stimulation method is provided therapeutic benefits for treatment chronic pain, essential tremor, Parkinsons disease, major depression, and neurological movement disorder syndrome (dystonia). One area in which advancements have been recently made is in controlling the movement and navigation of animals i…
Targeted electrical stimulation of the brain perturbs neural networks and modulates their rhythmic activity both at the site of stimulation and at remote brain regions. Understanding, or even predicting, this neuromodulatory effect is crucial for any therapeutic use of brain stimulation. To this end, we analyzed the stimulation responses in 131 stimulation sessions across 66 patients with focal epilepsy recorded through intracranial EEG (iEEG). We considered functional and structural connectivity features as predictors of the response at every iEEG contact. Taking advantage of multiple recordi…
Conventional deep brain stimulation (DBS) of basal ganglia uses high-frequency regular electrical pulses to treat Parkinsonian motor symptoms and has a series of limitations. Relatively new and not yet clinically tested optogenetic stimulation is an effective experimental stimulation technique to affect pathological network dynamics. We compared the effects of electrical and optogenetic stimulation of the basal ganglia on the pathological parkinsonian rhythmic neural activity. We studied the network response to electrical stimulation and excitatory and inhibitory optogenetic stimulations. Diff…
Essential Tremor is the most common neurological movement disorder. This progressive disease causes uncontrollable rhythmic motions -most often affecting the patient's dominant upper extremity- that occur during volitional movement and make it difficult for the patient to perform everyday tasks. Medication may also become ineffective as the disorder progresses. For many patients, deep brain stimulation (DBS) of the thalamus is an effective means of treating this condition when medication fails. In current use, however, clinicians set the patient's stimulator to apply stimulation at all times- …
Neurostimulation technologies have seen a recent surge in interest from the neuroscience and controls communities alike due to their proven potential to treat conditions such as Parkinson's Disease, and depression. The provided stimulation can be of different types, such as electric, and optogenetic, and is generally applied to a specific region of the brain in order to drive the local and/or global dynamics to a desired state of (in)activity. However, an underlying theoretical understanding of the efficacy of neurostimulation is still lacking. From a control-theoretic perspective, it is impor…
Nerve stimulation via micro-electrode implants is one of the neurostimulation approaches which is used frequently in the medical treatment of some brain disorders, neural prosthetics, brain-machine interfaces and also in the cyborg. In this method, the electrical stimulation signal can be categorized by the frequency band: low frequency, high frequency, and ultra-high frequency. The stimulation should be less destructive, more smooth, and controllable. In this article, we present a brief description of the mechanism underlying the ultra-high frequency stimulation. In the flowing, from an infor…
Brain stimulation is a core method in neuroscience. Numerous non-invasive brain stimulation (NIBS) techniques are currently in use in basic and clinical research, and recent advances promise the ability to non-invasively access deep brain structures. While encouraging, there is a surprising gap in our understanding of precisely how NIBS perturbs neural activity throughout an interconnected network, and how such perturbed neural activity ultimately links to behaviour. In this review, we will consider why non-human primate (NHP) models of NIBS are ideally situated to address this gap in knowledg…
The availability of technological means to enhance and repair human cognitive function raises questions about the perceived morality of their use. In this study, we administered a survey to the public in which subjects were asked to report how willing they would be to enhance and/or repair specific cognitive abilities. Among 894 responders, we found that subjects were more willing to use technologies to repair other people than themselves, and especially to enhance or repair functions more "core" to authentic identity in others. Subjects' ratings of the moral acceptability of specific uses was…
Imaging studies suggest that the functional connectivity patterns of resting state networks (RS-networks) reflect underlying structural connectivity (SC). If the connectome constrains how brain areas are functionally connected, the stimulation of specific brain areas should produce a characteristic wave of activity ultimately resolving into RS-networks. To systematically test this hypothesis, we use a connectome-based network model of the human brain with detailed realistic SC. We systematically activate all possible thalamic and cortical areas with focal stimulation patterns and confirm that …
At the macroscale, the brain operates as a network of interconnected neuronal populations, which display rhythmic dynamics that support interareal communication. Understanding how stimulation of a particular brain area impacts such concerted activity is important for gaining basic insights into brain function and for developing neuromodulation as a therapeutic tool. However, it remains difficult to predict the downstream effects of focal stimulation. Specifically, little is known about how the collective oscillatory regime of network activity may affect the outcomes of regional perturbations o…
Nagoya Institute of Technology（名古屋工业大学）；Suez Canal University（苏伊士运河大学）
Electro-stimulation or modulation of deep brain regions is commonly used in clinical procedures for the treatment of several nervous system disorders. In particular, transcranial direct current stimulation (tDCS) is widely used as an affordable clinical application that is applied through electrodes attached to the scalp. However, it is difficult to determine the amount and distribution of the electric field (EF) in the different brain regions due to anatomical complexity and high inter-subject variability. Personalized tDCS is an emerging clinical procedure that is used to tolerate electrode …
University of Montreal（蒙特利尔大学）；Concordia University（康考迪亚大学）
Closed-loop brain stimulation refers to capturing neurophysiological measures such as electroencephalography (EEG), quickly identifying neural events of interest, and producing auditory, magnetic or electrical stimulation so as to interact with brain processes precisely. It is a promising new method for fundamental neuroscience and perhaps for clinical applications such as restoring degraded memory function; however, existing tools are expensive, cumbersome, and offer limited experimental flexibility. In this article, we propose the Portiloop, a deep learning-based, portable and low-cost close…
Electrical brain stimulation is currently being investigated as a therapy for neurological disease. However, opportunities to optimize such therapies are challenged by the fact that the beneficial impact of focal stimulation on both neighboring and distant regions is not well understood. Here, we use network control theory to build a model of brain network function that makes predictions about how stimulation spreads through the brain's white matter network and influences large-scale dynamics. We test these predictions using combined electrocorticography (ECoG) and diffusion weighted imaging (…
Objective: Closed-loop deep brain stimulation (DBS) may improve current clinical DBS treatment for neurological movement disorders, but control algorithms may perform differently across patients. New metrics are needed for comparing and evaluating closed-loop algorithm performance that address the specific needs of closed-loop neuromodulation controllers. Approach: A metric is proposed for system performance that includes normalized terms that can be used to compare algorithm performance for a patient. This metric was evaluated using two closed-loop control algorithms that were tested in patie…
We study the extent to which vibrotactile stimuli delivered to the head of a subject can serve as a platform for a brain computer interface (BCI) paradigm. Six head positions are used to evoke combined somatosensory and auditory (via the bone conduction effect) brain responses, in order to define a multimodal tactile and auditory brain computer interface (taBCI). Experimental results of subjects performing online taBCI, using stimuli with a moderately fast inter-stimulus interval (ISI), validate the taBCI paradigm, while the feasibility of the concept is illuminated through information transfe…
Deep brain stimulation(DBS)has become an effective intervention for advanced Parkinson's disease, but the exact mechanism of DBS is still unclear. In this review, we discuss the history of DBS, the anatomy and internal architecture of the basal ganglia(BG), the abnormal pathological changes of the BG in Parkinson's disease, and how computational models can help understand and advance DBS. We also describe two types of models:mathematical theoretical models and clinical predictive models. Mathematical theoretical models simulate neurons or neural networks of BG to shed light on the mechanistic …
This study demonstrates the capability of external signal recording into memory and the reproduction of memory trace of this pattern in EEG by direct AC electrical stimulation of rat cerebral cortex. Additionally, we examine shifts of the DC potential level related to these phenomena. We show that in the course of memory trace reproduction, consecutive phases of engram activation and relaxation are registered and accompanied by corresponding negative and positive DC shifts. The observed electrophysiological changes may reflect consecutive activation and inhibition phases of neural ensembles pa…
Background: Direct electrical stimulation of the brain through intracranial electrodes is currently used to probe the epileptic brain as part of pre-surgical evaluation, and it is also being considered for therapeutic treatments through neuromodulation. It is still unknown, however, how consistent intracranial direct electrical stimulation responses are across sessions, to allow effective neuromodulation design. Objective: To investigate the cross-session consistency of the electrophysiological effect of electrical stimulation delivered through intracranial EEG. Methods: We analysed data from …
Non-invasive brain stimulation, such as transcranial alternating current stimulation (tACS) provides a powerful tool to directly modulate brain oscillations that mediate complex cognitive processes. While the body of evidence about the effect of tACS on behavioral and cognitive performance is constantly growing, those studies fail to address the importance of subject- specific stimulation protocols. With this study here, we set the foundation to combine tACS with a recently presented framework that utilizes real-time fRMI and Bayesian optimization in order to identify the most optimal tACS pro…
We present an error-controlled mesh refinement procedure for needle insertion simulation and apply it to the simulation of electrode implantation for deep brain stimulation, including brain shift. Our approach enables to control the error in the computation of the displacement and stress fields around the needle tip and needle shaft by suitably refining the mesh, whilst maintaining a coarser mesh in other parts of the domain. We demonstrate through academic and practical examples that our approach increases the accuracy of the displacement and stress fields around the needle without increasing…
Objective: A major challenge in designing closed-loop brain-computer interfaces is finding optimal stimulation patterns as a function of ongoing neural activity for different subjects and objectives. Approach: To achieve goal-directed closed-loop neurostimulation, we propose "neural co-processors" which use artificial neural networks and deep learning to learn optimal closed-loop stimulation policies, shaping neural activity and bridging injured neural circuits for targeted repair and rehabilitation. The co-processor adapts the stimulation policy as the biological circuit itself adapts to the …
Suppression of excessively synchronous beta-band oscillatory activity in the brain is believed to suppress hypokinetic motor symptoms of Parkinson's disease. Recently, a lot of interest has been devoted to desynchronizing delayed feedback deep brain stimulation (DBS). This type of synchrony control was shown to destabilize the synchronized state in networks of simple model oscillators as well as in networks of coupled model neurons. However, the dynamics of the neural activity in Parkinson's disease exhibits complex intermittent synchronous patterns, far from the idealized synchronous dynamics…
University of Denver（丹佛大学）；Kaiser Hospital（凯撒医院）；Colorado Neurological Institute（科罗拉多神经学研究所）
This paper presents the results of our recent work on studying the effects of deep brain stimulation (DBS) and medication on the dynamics of brain local field potential (LFP) signals used for behavior analysis of patients with Parkinson s disease (PD). DBS is a technique used to alleviate the severe symptoms of PD when pharmacotherapy is not very effective. Behavior recognition from the LFP signals recorded from the subthalamic nucleus (STN) has application in developing closed-loop DBS systems, where the stimulation pulse is adaptively generated according to subjects performing behavior. Most…
The human brain exhibits a relatively stable spatiotemporal organization that supports brain function and can be manipulated via local brain stimulation. Such perturbations to local cortical dynamics are globally integrated by distinct neural systems. However, it remains unclear how and why local changes in neural activity affect large-scale system dynamics. Here, we briefly review empirical and computational studies addressing how localized perturbations affect brain activity. We then systematically analyze a model of large-scale brain dynamics, assessing how localized changes in brain activi…
The paper presents results from a computational neuroscience study conducted to test vibrotactile stimuli delivered to subject fingertips and head areas in order to evoke the somatosensory brain responses utilized in a haptic brain computer interface (hBCI) paradigm. We present the preliminary and very encouraging results, with subjects conducting online hBCI interfacing experiments, ranging from 40% to 90% with a very fast inter-stimulus-interval (ISI) of 250ms. The presented results confirm our hypothesis that the hBCI paradigm concept is valid and it allows for rapid stimuli presentation in…
Deep brain stimulation (DBS) can alleviate the movement disorders like Parkinson's disease (PD). Indeed, it is known that aberrant beta (13-30Hz)oscillations and the loss of dopaminergic neurons in the basal ganglia-thalamus (BGTH) and cortex characterize the akinesia symptoms of PD. However, the relevant biophysical mechanism behind this process still remains unclear. Based on the prior striatal inhibitory model, we propose an extended BGTH model incorporating medium spine neurons (MSNs) and fast-spiking interneurons (FSIs) along with the effect of DBS. We are focusing in this paper on an ope…
High precision neuromodulation is a powerful tool to decipher neurocircuits and treat neurological diseases. Current non-invasive neuromodulation methods offer limited precision at the millimeter level. Here, we report optically-generated focused ultrasound (OFUS) for non-invasive brain stimulation with ultrahigh precision. OFUS is generated by a soft optoacoustic pad (SOAP) fabricated through embedding candle soot nanoparticles in a curved polydimethylsiloxane film. SOAP generates a transcranial ultrasound focus at 15 MHz with an ultrahigh lateral resolution of 83 um, which is two orders of m…
Mental disorders may exhibit pathological brain rhythms and neurostimulation promises to alleviate of patients' symptoms by modifying these rhythms. Today, most neurostimulation schemes are open-loop, i.e. administer experimental stimulation protocols independent of the patients brain activity which may yield a sub-optimal treatment. We propose a closed-loop feedback control scheme estimating an optimal stimulation based on observed brain activity. The optimal stimulation is chosen according to a user-defined target frequency distribution, which permits frequency tuning of the brain activity i…
Low-intensity ultrasonic waves can remotely and nondestructively excite central nervous system (CNS) neurons. While diverse applications for this effect are already emerging, the biophysical transduction mechanism underlying this excitation remains unclear. Recently, we suggested that ultrasound-induced intramembrane cavitation within the bilayer membrane could underlie the biomechanics of a range of observed acoustic bioeffects. In this paper, we show that, in CNS neurons, ultrasound-induced cavitation of these nanometric bilayer sonophores can induce a complex mechanoelectrical interplay lea…
University of Freiburg（弗赖堡大学）；Radboud University（拉德堡德大学）
The decoding of brain signals recorded via, e.g., an electroencephalogram, using machine learning is key to brain-computer interfaces (BCIs). Stimulation parameters or other experimental settings of the BCI protocol typically are chosen according to the literature. The decoding performance directly depends on the choice of parameters, as they influence the elicited brain signals and optimal parameters are subject-dependent. Thus a fast and automated selection procedure for experimental parameters could greatly improve the usability of BCIs. We evaluate a standalone random search and a combined…
A large scale computational model of the basal ganglia (BG) network is proposed to describes movement disorder including deep brain stimulation (DBS). The model of this complex network considers four areas of the basal ganglia network: the subthalamic nucleus (STN) as target area of DBS, globus pallidus, both pars externa and pars interna (GPe-GPi), and the thalamus (THA). Parkinsonian conditions are simulated by assuming reduced dopaminergic input and corresponding pronounced inhibitory or disinhibited projections to GPe and GPi. Macroscopic quantities can be derived which correlate closely t…
