People with Type 1 diabetes (T1D) require regular exogenous infusion of insulin to maintain their blood glucose concentration in a therapeutically adequate target range. Although the artificial pancreas and continuous glucose monitoring have been proven to be effective in achieving closed-loop control, significant challenges still remain due to the high complexity of glucose dynamics and limitations in the technology. In this work, we propose a novel deep reinforcement learning model for single-hormone (insulin) and dual-hormone (insulin and glucagon) delivery. In particular, the delivery strategies are developed by double Q-learning with dilated recurrent neural networks. For designing and testing purposes, the FDA-accepted UVA/Padova Type 1 simulator was employed. First, we performed long-term generalized training to obtain a population model. Then, this model was personalized with a small data-set of subject-specific data. In silico results show that the single and dual-hormone delivery strategies achieve good glucose control when compared to a standard basal-bolus therapy with low-glucose insulin suspension. Specifically, in the adult cohort (n=10), percentage time in target range [70, 180] mg/dL improved from 77.6% to 80.9% with single-hormone control, and to $85.6\%$ with dual-hormone control. In the adolescent cohort (n=10), percentage time in target range improved from 55.5% to 65.9% with single-hormone control, and to 78.8% with dual-hormone control. In all scenarios, a significant decrease in hypoglycemia was observed. These results show that the use of deep reinforcement learning is a viable approach for closed-loop glucose control in T1D.
We propose a dual-hormone delivery strategy by exploiting deep reinforcement learning (RL) for people with Type 1 Diabetes (T1D). Specifically, double dilated recurrent neural networks (RNN) are used to learn the hormone delivery strategy, trained by a variant of Q-learning, whose inputs are raw data of glucose \& meal carbohydrate and outputs are dual-hormone (insulin and glucagon) delivery. Without prior knowledge of the glucose-insulin metabolism, we run the method on the UVA/Padova simulator. Hundreds days of self-play are performed to obtain a generalized model, then importance sampling i…
University of Bristol（布里斯托尔大学）；The Open University（开放大学）；Cardiff University（卡迪夫大学）
In this paper we investigate the use of model-based reinforcement learning to assist people with Type 1 Diabetes with insulin dose decisions. The proposed architecture consists of multiple Echo State Networks to predict blood glucose levels combined with Model Predictive Controller for planning. Echo State Network is a version of recurrent neural networks which allows us to learn long term dependencies in the input of time series data in an online manner. Additionally, we address the quantification of uncertainty for a more robust control. Here, we used ensembles of Echo State Networks to capt…
GUIDE: Reinforcement Learning for Behavioral Action Support in Type 1 Diabetes
College of Health Solutions, Arizona State University（亚利桑那州立大学健康解决方案学院）；School of Computing and Augmented Intelligence, Arizona State University（亚利桑那州立大学计算与增强智能学院）；School for Engineering of Matter Transport and Energy, Arizona State University（亚利桑那州立大学物质传输与能源工程学院）；Institute for Foundation of Machine Learning, The University of Texas at Austin（德克萨斯大学奥斯汀分校机器学习基础研究所）；Sony AI（索尼人工智能）
Institute for Basic Science (IBS)（基础科学研究院）；Seoul National University（首尔大学）；Pohang University of Science and Technology（浦项科技大学）
Machine learning shows remarkable success for recognizing patterns in data. Here we apply the machine learning (ML) for the diagnosis of early stage diabetes, which is known as a challenging task in medicine. Blood glucose levels are tightly regulated by two counter-regulatory hormones, insulin and glucagon, and the failure of the glucose homeostasis leads to the common metabolic disease, diabetes mellitus. It is a chronic disease that has a long latent period the complicates detection of the disease at an early stage. The vast majority of diabetics result from that diminished effectiveness of…
Rambam Health Care Campus（兰巴姆医疗园区）；Technion, Israel Institute of Technology（以色列理工学院）；Microsoft Research（微软研究院）
Regular physical activity is known to be beneficial to people suffering from diabetes type 2. Nevertheless, most such people are sedentary. Smartphones create new possibilities for helping people to adhere to their physical activity goals, through continuous monitoring and communication, coupled with personalized feedback. We provided 27 sedentary diabetes type 2 patients with a smartphone-based pedometer and a personal plan for physical activity. Patients were sent SMS messages to encourage physical activity between once a day and once per week. Messages were personalized through a Reinforcem…
CNRS-LIMSI（法国国家科学研究中心-LIMSI）；Université Paris-Saclay（巴黎萨克雷大学）；Université Paris 8（巴黎第八大学）；Samovar；CNRS（法国国家科学研究中心）；Télécom SudParis（南巴黎电信学院）；Institut Polytechnique de Paris（巴黎综合理工学院）
Due to the sensitive nature of diabetes-related data, preventing them from being shared between studies, progress in the field of glucose prediction is hard to assess. To address this issue, we present GLYFE (GLYcemia Forecasting Evaluation), a benchmark of machine-learning-based glucose-predictive models. To ensure the reproducibility of the results and the usability of the benchmark in the future, we provide extensive details about the data flow. Two datasets are used, the first comprising 10 in-silico adults from the UVA/Padova Type 1 Diabetes Metabolic Simulator (T1DMS) and the second bein…
People with type 1 diabetes (T1D) lack the ability to produce the insulin their bodies need. As a result, they must continually make decisions about how much insulin to self-administer to adequately control their blood glucose levels. Longitudinal data streams captured from wearables, like continuous glucose monitors, can help these individuals manage their health, but currently the majority of the decision burden remains on the user. To relieve this burden, researchers are working on closed-loop solutions that combine a continuous glucose monitor and an insulin pump with a control algorithm i…
This work proposes a smartphone video-based approach for the estimation of blood glucose in a non-invasive way. Videos using smartphone camera are collected from the tip of the subjects finger and the frames are subsequently converted into Photoplethysmography (PPG) waveform. Gaussian filter along with Asymmetric Least Square methods have been applied on the PPG signals to remove the high-frequency noise, optical and motion interferences. Different signal features such as Systolic and Diastolic Peaks, the time difference between consecutive peaks (DelT), First Derivative peaks, and Second deri…
Technical University of Denmark（丹麦技术大学）；Novo Nordisk A/S（诺和诺德）
Continuous Glucose Monitoring (CGM) has enabled important opportunities for diabetes management. This study explores the use of CGM data as input for digital decision support tools. We investigate how Recurrent Neural Networks (RNNs) can be used for Short Term Blood Glucose (STBG) prediction and compare the RNNs to conventional time-series forecasting using Autoregressive Integrated Moving Average (ARIMA). A prediction horizon up to 90 min into the future is considered. In this context, we evaluate both population-based and patient-specific RNNs and contrast them to patient-specific ARIMA mode…
Research in diabetes, especially when it comes to building data-driven models to forecast future glucose values, is hindered by the sensitive nature of the data. Because researchers do not share the same data between studies, progress is hard to assess. This paper aims at comparing the most promising algorithms in the field, namely Feedforward Neural Networks (FFNN), Long Short-Term Memory (LSTM) Recurrent Neural Networks, Extreme Learning Machines (ELM), Support Vector Regression (SVR) and Gaussian Processes (GP). They are personalized and trained on a population of 10 virtual children from t…
In the case of diabetes, fingertip pricking for a blood sample is inconvenient for glucose measurement. Invasive approaches like laboratory test and one-touch glucometer enhance the risk of blood-related infections. To mitigate this important issue, in the current paper, we propose a novel Internet-of-Medical-Things (IoMT) enabled edge-device for precise, non-invasive blood glucose measurement. In this work, a near-infrared (NIR) spectroscopic technique using two wavelengths (940 nm, 1300 nm) is taken to detect the glucose molecule from human blood. The novel device called iGLU is based on NIR…
