Deployment of machine learning (ML) algorithms in production for extended periods of time has uncovered new challenges such as monitoring and management of real-time prediction quality of a model in the absence of labels. However, such tracking is imperative to prevent catastrophic business outcomes resulting from incorrect predictions. The scale of these deployments makes manual monitoring prohibitive, making automated techniques to track and raise alerts imperative. We present a framework, ML Health, for tracking potential drops in the predictive performance of ML models in the absence of labels. The framework employs diagnostic methods to generate alerts for further investigation. We develop one such method to monitor potential problems when production data patterns do not match training data distributions. We demonstrate that our method performs better than standard "distance metrics", such as RMSE, KL-Divergence, and Wasserstein at detecting issues with mismatched data sets. Finally, we present a working system that incorporates the ML Health approach to monitor and manage ML deployments within a realistic full production ML lifecycle.
Automatic detection of emotion has the potential to revolutionize mental health and wellbeing. Recent work has been successful in predicting affect from unimodal electrocardiogram (ECG) data. However, to be immediately relevant for real-world applications, physiology-based emotion detection must make use of ubiquitous photoplethysmogram (PPG) data collected by affordable consumer fitness trackers. Additionally, applications of emotion detection in healthcare settings will require some measure of uncertainty over model predictions. We present here a Bayesian deep learning model for end-to-end c…
FedFiTS: Fitness-Selected, Slotted Client Scheduling for Trustworthy Federated Learning in Healthcare AI
Department of Computer Science University of the Western Cape（计算机科学系，西开普敦大学）；Department of Computer Science Missouri University of Science and Technology（计算机科学系，密苏里科学与技术大学）
University of Cambridge（剑桥大学）；MRC Epidemiology Unit（英国医学研究委员会流行病学组）
Deep learning models have shown great promise in various healthcare applications. However, most models are developed and validated on small-scale datasets, as collecting high-quality (gold-standard) labels for health applications is often costly and time-consuming. As a result, these models may suffer from overfitting and not generalize well to unseen data. At the same time, an extensive amount of data with imprecise labels (silver-standard) is starting to be generally available, as collected from inexpensive wearables like accelerometers and electrocardiography sensors. These currently underu…
Columbia University（哥伦比亚大学）；Data Science Institute, Columbia University（哥伦比亚大学数据科学研究所）；Clue by BioWink（BioWink公司旗下Clue项目）；Department of Anthropology, Indiana University（印第安纳大学人类学系）；Department of Biomedical Informatics, Columbia University（哥伦比亚大学生物医学信息学系）
Mobile health (mHealth) apps such as menstrual trackers provide a rich source of self-tracked health observations that can be leveraged for health-relevant research. However, such data streams have questionable reliability since they hinge on user adherence to the app. Therefore, it is crucial for researchers to separate true behavior from self-tracking artifacts. By taking a machine learning approach to modeling self-tracked cycle lengths, we can both make more informed predictions and learn the underlying structure of the observed data. In this work, we propose and evaluate a hierarchical, g…
Stanford（斯坦福大学）；Clue by BioWink GmbH（Clue by BioWink有限责任公司）；Chan-Zuckerberg Biohub（陈-扎克伯格生物中心）
Predicting pregnancy has been a fundamental problem in women's health for more than 50 years. Previous datasets have been collected via carefully curated medical studies, but the recent growth of women's health tracking mobile apps offers potential for reaching a much broader population. However, the feasibility of predicting pregnancy from mobile health tracking data is unclear. Here we develop four models -- a logistic regression model, and 3 LSTM models -- to predict a woman's probability of becoming pregnant using data from a women's health tracking app, Clue by BioWink GmbH. Evaluating ou…
The huge wealth of data in the health domain can be exploited to create models that predict development of health states over time. Temporal learning algorithms are well suited to learn relationships between health states and make predictions about their future developments. However, these algorithms: (1) either focus on learning one generic model for all patients, providing general insights but often with limited predictive performance, or (2) learn individualized models from which it is hard to derive generic concepts. In this paper, we present a middle ground, namely parameterized dynamical…
Indraprastha Institute of Information Technology, Delhi (IIIT-D)（德里英德拉普拉斯塔信息技术学院（IIIT-D））；Software Analytics Research Lab (SARL)（软件分析研究实验室（SARL））
Issue Tracking Systems (ITS) such as Bugzilla can be viewed as Process Aware Information Systems (PAIS) generating event-logs during the life-cycle of a bug report. Process Mining consists of mining event logs generated from PAIS for process model discovery, conformance and enhancement. We apply process map discovery techniques to mine event trace data generated from ITS of open source Firefox browser project to generate and study process models. Bug life-cycle consists of diversity and variance. Therefore, the process models generated from the event-logs are spaghetti-like with large number o…
Graz University of Technology（格拉茨技术大学）；University of Graz（格拉茨大学）
Recording muscle tendon junction displacements during movement, allows separate investigation of the muscle and tendon behaviour, respectively. In order to provide a fully-automatic tracking method, we employ a novel deep learning approach to detect the position of the muscle tendon junction in ultrasound images. We utilize the attention mechanism to enable the network to focus on relevant regions and to obtain a better interpretation of the results. Our data set consists of a large cohort of 79 healthy subjects and 28 subjects with movement limitations performing passive full range of motion …
National University of Sciences and Technology（国立科技大学）
We analyze damage propagation modeling of turbo-engines in a data-driven approach. We investigate subspace tracking assuming a low dimensional manifold structure and a static behavior during the healthy state of the machines. Our damage propagation model is based on the deviation of the data from the static behavior and uses the notion of health index as a measure of the condition. Hence, we incorporate condition-based maintenance and estimate the remaining useful life based on the current and previous health indexes. This paper proposes an algorithm that adapts well to the dynamics of the dat…
Real-Time Magnetic Tracking and Diagnosis of COVID-19 via Machine Learning
University of South Florida（南佛罗里达大学）；Hue Central Hospital（顺化中央医院）；Riga Stradinš University（里加斯特拉迪宁什大学）；Vietnam National University（越南国家大学）；University Medical Center（大学医学中心）；Traditional Medicine Hospital, Ministry of Public Security（公安部传统医学医院）
A Comparison of Object Detection and Phrase Grounding Models in Chest X-ray Abnormality Localization using Eye-tracking Data
Scientific Computing and Imaging Institute, University of Utah（犹他大学科学计算与成像研究所）；Kahlert School of Computing, University of Utah（犹他大学卡勒特计算学院）；University of Utah（犹他大学）
With the advancement of technologies, market wearables are becoming increasingly popular with a range of services, including providing access to bank accounts, accessing cars, monitoring patients remotely, among several others. However, often these wearables collect various sensitive personal information of a user with no to limited authentication, e.g., knowledge-based external authentication techniques, such as PINs. While most of these external authentication techniques suffer from multiple limitations, including recall burden, human errors, or biases, researchers have started using various…
Emory School of Medicine（埃默里医学院）；Georgia Institute of Technology（佐治亚理工学院）
The expanding usage of complex machine learning methods like deep learning has led to an explosion in human activity recognition, particularly applied to health. In particular, as part of a larger body sensor network system, face and full-body analysis is becoming increasingly common for evaluating health status. However, complex models which handle private and sometimes protected data, raise concerns about the potential leak of identifiable data. In this work, we focus on the case of a deep network model trained on images of individual faces. Full-face video recordings taken from 493 individu…
i-Mask: An Intelligent Mask for Breath-Driven Activity Recognition
Department of Computer Science and Engineering, Indian Institute of Technology Patna, Bihar, India（计算机科学与工程系，印度理工学院帕纳布分校，比哈尔邦，印度）；Dhirubhai Ambani Institute of Information and Communication Technology (DA-IICT), Gandhinagar, Gujarat, India（迪鲁巴希·阿姆巴尼信息与通信技术研究所（DA-IICT），古吉拉特邦，古吉拉特邦，印度）
Enhancing ALS Progression Tracking with Semi-Supervised ALSFRS-R Scores Estimated from Ambient Home Health Monitoring
Institute for Data Science and Informatics（数据科学与信息学研究所）；Department of Occupational Therapy（职业治疗系）；Department of Biomedical Informatics, Biostatistics, and Medical Epidemiology（生物医学信息学、生物统计学和医学流行病学系）；University of Missouri（密苏里大学）
本研究提出半监督回归模型，利用家庭传感器数据估计 ALS 患者的 ALSFRS-R 量表衰退轨迹，发现将学习与伪标签技术匹配至功能领域特定的同质-异质特征可显著提升进展追踪的预测准确性。
University of Cambridge（剑桥大学）；Alan Turing Institute（艾伦·图灵研究所）
To date, research on sensor-equipped mobile devices has primarily focused on the purely supervised task of human activity recognition (walking, running, etc), demonstrating limited success in inferring high-level health outcomes from low-level signals, such as acceleration. Here, we present a novel self-supervised representation learning method using activity and heart rate (HR) signals without semantic labels. With a deep neural network, we set HR responses as the supervisory signal for the activity data, leveraging their underlying physiological relationship. We evaluate our model in the lar…
