We are developing a virtual coaching system that helps patients adhere to behavior change interventions (BCI). Our proposed system predicts whether a patient will perform the targeted behaviour and uses counterfactual examples with feature control to guide personalisation of BCI. We use simulated patient data with varying levels of receptivity to intervention to arrive at the study design which would enable evaluation of our system.
Mobile health (mHealth) technologies empower patients to adopt/maintain healthy behaviors in their daily lives, by providing interventions (e.g. push notifications) tailored to the user's needs. In these settings, without intervention, human decision making may be impaired (e.g. valuing near term pleasure over own long term goals). In this work, we formalize this relationship with a framework in which the user optimizes a (potentially impaired) Markov Decision Process (MDP) and the mHealth agent intervenes on the user's MDP parameters. We show that different types of impairments imply differen…
Policy Optimization for Personalized Interventions in Behavioral Health
Stern School of Business, New York University（纽约大学斯特恩商学院）；Telfer School of Management, University of Ottawa（渥太华大学泰尔弗管理学院）；Sloan School of Management, Massachusetts Institute of Technology（麻省理工学院斯隆管理学院）
In many public health settings, it is important for patients to adhere to health programs, such as taking medications and periodic health checks. Unfortunately, beneficiaries may gradually disengage from such programs, which is detrimental to their health. A concrete example of gradual disengagement has been observed by an organization that carries out a free automated call-based program for spreading preventive care information among pregnant women. Many women stop picking up calls after being enrolled for a few months. To avoid such disengagements, it is important to provide timely intervent…
Bill and Melinda Gates Foundation（比尔及梅琳达·盖茨基金会）；Universidad de Barcelona（巴塞罗那大学）
Malaria can be prevented, diagnosed, and treated; however, every year, there are more than 200 million cases and 200.000 preventable deaths. Malaria remains a pressing public health concern in low- and middle-income countries, especially in sub-Saharan Africa. We describe how by means of mobile health applications, machine-learning-based adaptive interventions can strengthen malaria surveillance and treatment adherence, increase testing, measure provider skills and quality of care, improve public health by supporting front-line workers and patients (e.g., by capacity building and encouraging b…
Harvard University（哈佛大学）；University of Virginia（弗吉尼亚大学）
We propose and study Collpasing Bandits, a new restless multi-armed bandit (RMAB) setting in which each arm follows a binary-state Markovian process with a special structure: when an arm is played, the state is fully observed, thus "collapsing" any uncertainty, but when an arm is passive, no observation is made, thus allowing uncertainty to evolve. The goal is to keep as many arms in the "good" state as possible by planning a limited budget of actions per round. Such Collapsing Bandits are natural models for many healthcare domains in which workers must simultaneously monitor patients and deli…
Adaptive Multi-Agent Deep Reinforcement Learning for Timely Healthcare Interventions
University of Southern Queensland（南昆士兰大学）；Wuhan University of Technology（武汉理工大学）；Lingnan University（岭南大学）；Hong Kong Baptist University（香港浸会大学）；Huazhong University of Science and Technology（华中科技大学）
UC San Diego（加州大学圣地亚哥分校）；Maternity Foundation（产妇基金会）
Mobile health apps provide a unique means of collecting data that can be used to deliver adaptive interventions.The predicted outcomes considerably influence the selection of such interventions. Recourse via counterfactuals provides tangible mechanisms to modify user predictions. By identifying plausible actions that increase the likelihood of a desired prediction, stakeholders are afforded agency over their predictions. Furthermore, recourse mechanisms enable counterfactual reasoning that can help provide insights into candidates for causal interventional features. We demonstrate the feasibil…
Robert Bosch Centre for Data Science and Artificial Intelligence（罗伯特·博世数据科学与人工智能中心）；IIT Madras（印度理工学院马德拉斯分校）；Google Research（谷歌研究院）；Harvard University（哈佛大学）；ARMMAN
India has a maternal mortality ratio of 113 and child mortality ratio of 2830 per 100,000 live births. Lack of access to preventive care information is a major contributing factor for these deaths, especially in low resource households. We partner with ARMMAN, a non-profit based in India employing a call-based information program to disseminate health-related information to pregnant women and women with recent child deliveries. We analyze call records of over 300,000 women registered in the program created by ARMMAN and try to identify women who might not engage with these call programs that a…
Amazon（亚马逊）；University of Michigan（密歇根大学）；Radcliffe Institute, Harvard University（哈佛大学拉德克利夫研究所）
Increasing technological sophistication and widespread use of smartphones and wearable devices provide opportunities for innovative and highly personalized health interventions. A Just-In-Time Adaptive Intervention (JITAI) uses real-time data collection and communication capabilities of modern mobile devices to deliver interventions in real-time that are adapted to the in-the-moment needs of the user. The lack of methodological guidance in constructing data-based JITAIs remains a hurdle in advancing JITAI research despite the increasing popularity of JITAIs among clinical scientists. In this a…
Harvard University（哈佛大学）；University of Michigan（密歇根大学）；University of California, Los Angeles（加利福尼亚大学洛杉矶分校）
Online reinforcement learning (RL) algorithms are increasingly used to personalize digital interventions in the fields of mobile health and online education. Common challenges in designing and testing an RL algorithm in these settings include ensuring the RL algorithm can learn and run stably under real-time constraints, and accounting for the complexity of the environment, e.g., a lack of accurate mechanistic models for the user dynamics. To guide how one can tackle these challenges, we extend the PCS (Predictability, Computability, Stability) framework, a data science framework that incorpor…
Michael G. Foster School of Business, University of Washington（华盛顿大学迈克尔·G·福斯特商学院）；Kelley School of Business, Indiana University（印第安纳大学凯利商学院）
Online healthcare communities provide users with various healthcare interventions to promote healthy behavior and improve adherence. When faced with too many intervention choices, however, individuals may find it difficult to decide which option to take, especially when they lack the experience or knowledge to evaluate different options. The choice overload issue may negatively affect users' engagement in health management. In this study, we take a design-science perspective to propose a recommendation framework that helps users to select healthcare interventions. Taking into account that user…
Multimodal Ambivalence/Hesitancy Recognition in Videos for Personalized Digital Health Interventions
LIVIA, Dept. of Systems Engineering, ETS Montreal, Canada（ETS蒙特利尔大学系统工程系LIVIA实验室）；LIVIA, Dept. of Software and IT Engineering, ETS Montreal, Canada（ETS蒙特利尔大学软件与信息工程系LIVIA实验室）；Dept. of Health, Kinesiology, & Applied Physiology, Concordia University, Montreal, Canada（康科迪亚大学健康、运动科学与应用生理学系）；Montreal Behavioural Medicine Centre, CIUSSS Nord-de-l’Ile-de-Montréal, Canada（蒙特利尔行为医学中心，蒙特利尔北岛卫生与社会服务局）
We consider the actor-critic contextual bandit for the mobile health (mHealth) intervention. State-of-the-art decision-making algorithms generally ignore the outliers in the dataset. In this paper, we propose a novel robust contextual bandit method for the mHealth. It can achieve the conflicting goal of reducing the influence of outliers while seeking for a similar solution compared with the state-of-the-art contextual bandit methods on the datasets without outliers. Such performance relies on two technologies: (1) the capped-$\ell_{2}$ norm; (2) a reliable method to set the thresholding hyper…
University of Michigan（密歇根大学）；School of Information, University of Michigan（密歇根大学信息学院）；Harvard University（哈佛大学）
Due to the recent advancements in wearables and sensing technology, health scientists are increasingly developing mobile health (mHealth) interventions. In mHealth interventions, mobile devices are used to deliver treatment to individuals as they go about their daily lives. These treatments are generally designed to impact a near time, proximal outcome such as stress or physical activity. The mHealth intervention policies, often called just-in-time adaptive interventions, are decision rules that map an individual's current state (e.g., individual's past behaviors as well as current observation…
In the context of digital therapy interventions, such as internet-delivered Cognitive Behavioral Therapy (iCBT) for the treatment of depression and anxiety, extensive research has shown how the involvement of a human supporter or coach, who assists the person undergoing treatment, improves user engagement in therapy and leads to more effective health outcomes than unsupported interventions. Seeking to maximize the effects and outcomes of this human support, the research investigates how new opportunities provided through recent advances in the field of AI and machine learning (ML) can contribu…
Harvard T.H. Chan School of Public Health（哈佛陈曾熙公共卫生学院）；Wellframe Inc.（Wellframe公司）
It is not clear how to target patients who are most likely to benefit from digital care management programs ex-ante, a shortcoming of current risk score based approaches. This study focuses on defining impactability by identifying those patients most likely to benefit from technology enabled care management, delivered through a digital health platform, including a mobile app and clinician web dashboard. Anonymized insurance claims data were used from a commercially insured population across several U.S. states and combined with inferred sociodemographic data and data derived from the patient-h…
Prototype-Based Learning for Healthcare: A Demonstration of Interpretable AI
NEC Laboratories Europe GmbH（NEC欧洲实验室）；NEC Solution Innovators, Ltd.（NEC解决方案创新者有限公司）；Kurashiki Central Hospital Preventive Healthcare Plaza（Kurashiki中央医院预防保健广场）；Kurashiki Central Hospital（Kurashiki中央医院）
Using Adaptive Bandit Experiments to Increase and Investigate Engagement in Mental Health
University of Toronto（多伦多大学）；Northwestern University（西北大学）；Audacious Software；Mental Health America（美国心理健康组织）；Carleton College（卡尔顿学院）；University of Cambridge（剑桥大学）；Sapienza University of Rome（罗马大学）
Counterfactual Modeling with Fine-Tuned LLMs for Health Intervention Design and Sensor Data Augmentation
College of Health Solutions, Arizona State University（亚利桑那州立大学健康解决方案学院）；School of Computing and Augmented Intelligence, Arizona State University（亚利桑那州立大学计算与增强智能学院）；School of Nutritional Sciences and Wellness, University of Arizona（亚利桑那大学营养科学与健康学院）
Reinforcement Learning in Modern Biostatistics: Constructing Optimal Adaptive Interventions
Sapienza University of Rome（罗马大学）；University of Cambridge（剑桥大学）；University of Toronto（多伦多大学）；Duke-NUS Medical School（杜克-新加坡国立大学医学院）；National University of Singapore (NUS)（新加坡国立大学）；Duke University（杜克大学）
Contextual bandits often provide simple and effective personalization in decision making problems, making them popular tools to deliver personalized interventions in mobile health as well as other health applications. However, when bandits are deployed in the context of a scientific study -- e.g. a clinical trial to test if a mobile health intervention is effective -- the aim is not only to personalize for an individual, but also to determine, with sufficient statistical power, whether or not the system's intervention is effective. It is essential to assess the effectiveness of the interventio…
Delivering treatment recommendations via pervasive electronic devices such as mobile phones has the potential to be a viable and scalable treatment medium for long-term health behavior management. But active experimentation of treatment options can be time-consuming, expensive and altogether unethical in some cases. There is a growing interest in methodological approaches that allow an experimenter to learn and evaluate the usefulness of a new treatment strategy before deployment. We present the first development of a treatment recommender system for emotion regulation using real-world histori…
University of Chicago Booth School of Business（芝加哥大学布斯商学院）；University of Manchester（曼彻斯特大学）；University of Michigan（密歇根大学）；Institute for Social Research, University of Michigan（密歇根大学社会研究所）；Eli Broad College of Business, Michigan State University（密歇根州立大学伊莱·布罗德商学院）；Harvard University（哈佛大学）；Amazon（亚马逊公司）
