Accurate classification of sleep stages from less obtrusive sensor measurements such as the electrocardiogram (ECG) or photoplethysmogram (PPG) could enable important applications in sleep medicine. Existing approaches to this problem have typically used deep learning models designed and trained to operate on one or more specific input signals. However, the datasets used to develop these models often do not contain the same sets of input signals. Some signals, particularly PPG, are much less prevalent than others, and this has previously been addressed with techniques such as transfer learning. Additionally, only training on one or more fixed modalities precludes cross-modal information transfer from other sources, which has proved valuable in other problem domains. To address this, we introduce wav2sleep, a unified model designed to operate on variable sets of input signals during training and inference. After jointly training on over 10,000 overnight recordings from six publicly available polysomnography datasets, including SHHS and MESA, wav2sleep outperforms existing sleep stage classification models across test-time input combinations including ECG, PPG, and respiratory signals.
SleepBand: Single-Source Domain Generalization for Sleep Staging via Physiologically Structured Spectral Modeling
Laboratory of Intelligent Collaborative Computing, University of Electronic Science and Technology of China（电子科技大学智能协同计算实验室）；School of Information Science and Technology, University of Science and Technology of China（中国科学技术大学信息科学学院）；School of Cyber Science and Technology, University of Science and Technology of China（中国科学技术大学网络科学与技术学院）
The detection of human sleep stages is widely used in the diagnosis and intervention of neurological and psychiatric diseases. Some patients with deep brain stimulator implanted could have their neural activities recorded from the deep brain. Sleep stage classification based on deep brain recording has great potential to provide more precise treatment for patients. The accuracy and generalizability of existing sleep stage classifiers based on local field potentials are still limited. We proposed an applicable cross-modal transfer learning method for sleep stage classification with implanted de…
On Sleep-Wake Disorder Identification from Polysomnographic Representations
Faculty of Informatics, Università della Svizzera Italiana（瑞士意大利语区大学信息学院）；MeDiTech, SUPSI（瑞士应用科学与艺术大学医学技术学院）；Ente Ospedaliero Cantonale (EOC)（州立医院管理局）；University of Brescia（布雷西亚大学）；University of Bern（伯尔尼大学）
Omni-Sleep: A Sleep Foundation Model via Hierarchical Contrastive Learning of CNS-ANS Dynamics
Department of Biomedical Engineering, Southern University of Science and Technology（南方科技大学生物医学工程系）；Omni-Intelligence（全知智能）；Shenzhen Loop Area Institute（深圳河套学院）
In this work, a dense recurrent convolutional neural network (DRCNN) was constructed to detect sleep disorders including arousal, apnea and hypopnea using Polysomnography (PSG) measurement channels provided in the 2018 Physionet challenge database. Our model structure is composed of multiple dense convolutional units (DCU) followed by a bidirectional long-short term memory (LSTM) layer followed by a softmax output layer. The sleep events including sleep stages, arousal regions and multiple types of apnea and hypopnea are manually annotated by experts which enables us to train our proposed netw…
University of Oslo（奥斯陆大学）；National University of Singapore（新加坡国立大学）
Supervised machine learning applications in the health domain often face the problem of insufficient training datasets. The quantity of labelled data is small due to privacy concerns and the cost of data acquisition and labelling by a medical expert. Furthermore, it is quite common that collected data are unbalanced and getting enough data to personalize models for individuals is very expensive or even infeasible. This paper addresses these problems by (1) designing a recurrent Generative Adversarial Network to generate realistic synthetic data and to augment the original dataset, (2) enabling…
SemPSG: A Semantic Channel-Aware Foundation Model for Polysomnography Analysis
Center for Artificial Intelligence and Robotics, Hong Kong Institute of Science & Innovation, CAS（中国科学院香港创新研究院人工智能与机器人中心）；Shenzhen Institute for Advanced Study, University of Electronic Science and Technology of China（电子科技大学深圳高等研究院）；Technical University of Munich（慕尼黑工业大学）
Leveraging Foundational Models and Simple Fusion for Multi-modal Physiological Signal Analysis
Computer and Communication Engineering Department, Alexandria University（亚历山大大学计算机与通信工程系）；Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）
Newcastle University（纽卡斯尔大学）；Georgia Institute of Technology（佐治亚理工学院）
Sleep is a fundamental physiological process that is essential for sustaining a healthy body and mind. The gold standard for clinical sleep monitoring is polysomnography(PSG), based on which sleep can be categorized into five stages, including wake/rapid eye movement sleep (REM sleep)/Non-REM sleep 1 (N1)/Non-REM sleep 2 (N2)/Non-REM sleep 3 (N3). However, PSG is expensive, burdensome, and not suitable for daily use. For long-term sleep monitoring, ubiquitous sensing may be a solution. Most recently, cardiac and movement sensing has become popular in classifying three-stage sleep, since both m…
SleepMaMi: A Universal Sleep Foundation Model for Integrating Macro- and Micro-structures
Graduate School of Data Science, Seoul National University, Seoul, South Korea（首尔国立大学数据科学研究生院，韩国首尔）；Department of Biomedical Sciences, Seoul National University College of Medicine, Seoul, Republic of Korea（首尔国立大学医学院生物医学科学系，韩国首尔）；Obstructive Upper Airway Research (OUaR) Laboratory, Department of Pharmacology, Seoul National University College of Medicine, Seoul, Republic of Korea（首尔国立大学医学院药理学系阻塞性上气道研究（OUaR）实验室，韩国首尔）；Department of Otorhinolaryngology-Head and Neck Surgery, Seoul National University Hospital, Seoul, Republic of Korea（首尔国立大学医院耳鼻喉头颈外科系，韩国首尔）
sleep2vec: Unified Cross-Modal Alignment for Heterogeneous Nocturnal Biosignals
Five Seasons Medical（五 Seasons 医疗）；Tsinghua University（清华大学）；Beijing Key Laboratory for Sleep Breathing Disorder（北京睡眠呼吸障碍重点实验室）；Peking University（北京大学）；Fudan University（复旦大学）；Technical University of Munich（慕尼黑技术大学）
Sleep Brain and Cardiac Activity Predict Cognitive Flexibility and Conceptual Reasoning Using Deep Learning
Laboratoire d’imagerie, de vision et d’intelligence artificielle (LIVIA), École de technologie supérieure (ÉTS), Université du Québec（图像、视觉和人工智能实验室（LIVIA），工程学院（ÉTS），魁北克大学）；Ouraring Inc（Ouraring公司）；Center for Health Sciences, SRI International（健康科学中心，SRI国际）
SleepPPG-Net2: Deep learning generalization for sleep staging from photoplethysmography
The Faculty of Data and Decision Sciences, Technion, Israel Institute of Technology（以色列理工学院数据与决策科学学院）；Faculty of Biomedical Engineering, Technion, Israel Institute of Technology（以色列理工学院生物医学工程学院）
ASLSL: Adaptive shared latent structure learning with incomplete multi-modal physiological data for multi-dimensional emotional feature selection
School of Information Science and Technology, Beijing University of Technology, Beijing 100124, China（信息科学与技术学院，北京理工大学，北京）；School of Artificial Intelligence, Anhui University, Beijing 100124, China（人工智能学院，安徽大学，北京）
University of Copenhagen（哥本哈根大学）；Danish Center for Sleep Medicine（丹麦睡眠医学中心）；Rigshospitalet（丹麦国家医院）
Neural networks are becoming more and more popular for the analysis of physiological time-series. The most successful deep learning systems in this domain combine convolutional and recurrent layers to extract useful features to model temporal relations. Unfortunately, these recurrent models are difficult to tune and optimize. In our experience, they often require task-specific modifications, which makes them challenging to use for non-experts. We propose U-Time, a fully feed-forward deep learning approach to physiological time series segmentation developed for the analysis of sleep data. U-Tim…
AdaptStress: Online Adaptive Learning for Interpretable and Personalized Stress Prediction Using Multivariate and Sparse Physiological Signals
Engineering and Technology institute Groningen (ENTEG), University of Groningen（格罗宁根大学工程与技术学院（ENTEG））；Department of Human Movement Sciences, University Medical Center Groningen（格罗宁根大学医学中心人类运动科学系）
Sleep apnea is a common respiratory disorder characterized by breathing pauses during the night. Consequences of untreated sleep apnea can be severe. Still, many people remain undiagnosed due to shortages of hospital beds and trained sleep technicians. To assist in the diagnosis process, automated detection methods are being developed. Recent works have demonstrated that deep learning models can extract useful information from raw respiratory data and that such models can be used as a robust sleep apnea detector. However, trained sleep technicians take into account multiple sensor signals when…
Technical University of Denmark（丹麦技术大学）；Stanford University（斯坦福大学）；University Hospital of Copenhagen（哥本哈根大学医院）
Clinical sleep analysis require manual analysis of sleep patterns for correct diagnosis of sleep disorders. However, several studies have shown significant variability in manual scoring of clinically relevant discrete sleep events, such as arousals, leg movements, and sleep disordered breathing (apneas and hypopneas). We investigated whether an automatic method could be used for event detection and if a model trained on all events (joint model) performed better than corresponding event-specific models (single-event models). We trained a deep neural network event detection model on 1653 individ…
Technical University of Denmark（丹麦技术大学）；Stanford University（斯坦福大学）；University Hospital Copenhagen（哥本哈根大学医院）；Telecom ParisTech（巴黎高等电信学院）；Universite Paris-Saclay（巴黎-萨克雷大学）；Dreem（Dreem公司）
Much attention has been given to automatic sleep staging algorithms in past years, but the detection of discrete events in sleep studies is also crucial for precise characterization of sleep patterns and possible diagnosis of sleep disorders. We propose here a deep learning model for automatic detection and annotation of arousals and leg movements. Both of these are commonly seen during normal sleep, while an excessive amount of either is linked to disrupted sleep patterns, excessive daytime sleepiness impacting quality of life, and various sleep disorders. Our model was trained on 1,485 subje…
Finding Pre-Injury Patterns in Triathletes from Lifestyle, Recovery and Load Dynamics Features
Embedded Sensing Group ESG Institute of Computer Science in Vorarlberg ICV, University of St. Gallen HSG, Switzerland（嵌入式传感组 ESG 维尔阿尔贝格计算机科学研究所，圣加尔登大学 HSG，瑞士）
The regulation of the autonomic nervous system changes with the sleep stages causing variations in the physiological variables. We exploit these changes with the aim of classifying the sleep stages in awake or asleep using pulse oximeter signals. We applied a recurrent neural network to heart rate and peripheral oxygen saturation signals to classify the sleep stage every 30 seconds. The network architecture consists of two stacked layers of bidirectional gated recurrent units (GRUs) and a softmax layer to classify the output. In this paper, we used 5000 patients from the Sleep Heart Health Stu…
University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；Relativity（Relativity公司）；Harvard Medical School（哈佛医学院）；Beth Israel Deaconess Medical Center（贝斯以色列女执事医疗中心）
Background: Deep learning models have shown great success in automating tasks in sleep medicine by learning from carefully annotated Electroencephalogram (EEG) data. However, effectively utilizing a large amount of raw EEG remains a challenge. Objective: In this paper, we aim to learn robust vector representations from massive unlabeled EEG signals, such that the learned vectorized features (1) are expressive enough to replace the raw signals in the sleep staging task; and (2) provide better predictive performance than supervised models in scenarios of fewer labels and noisy samples. Methods: …
ULW-SleepNet: An Ultra-Lightweight Network for Multimodal Sleep Stage Scoring
School of Computer Science and Technology, Dalian University of Technology（大连理工大学计算机科学与技术学院）；Faculty of Information Technology, University of Jyväskylä（于韦斯屈莱大学信息科技学院）；Key Laboratory of Social Computing and Cognitive Intelligence, Ministry of Education（教育部社会计算与认知智能重点实验室）；School of Biomedical Engineering, Faculty of Medicine, Dalian University of Technology（大连理工大学生物医学工程学院）；Department of Computing Sciences, Tampere University（塔尔库大学计算机科学系）；Faculty of Medicine, University of Helsinki（赫尔辛基大学医学学院）
NYU Abu Dhabi（纽约大学阿布扎比分校）；NYU Grossman School of Medicine（纽约大学格罗斯曼医学院）
The healthcare domain is characterized by heterogeneous data modalities, such as imaging and physiological data. In practice, the variety of medical data assists clinicians in decision-making. However, most of the current state-of-the-art deep learning models solely rely upon carefully curated data of a single modality. In this paper, we propose a dynamic training approach to learn modality-specific data representations and to integrate auxiliary features, instead of solely relying on a single modality. Our preliminary experiments results for a patient phenotyping task using physiological data…
CNRS（法国国家科学研究中心）；University of Lille（里尔大学）；Centrale Lille（里尔中央理工学院）；IMT Nord Europe（法国北部高等矿业电信学院）；Institut Mines-Télécom（法国矿业电信学院）；INSERM（法国国家健康与医学研究院）
Existing multimodal stress/pain recognition approaches generally extract features from different modalities independently and thus ignore cross-modality correlations. This paper proposes a novel geometric framework for multimodal stress/pain detection utilizing Symmetric Positive Definite (SPD) matrices as a representation that incorporates the correlation relationship of physiological and behavioural signals from covariance and cross-covariance. Considering the non-linearity of the Riemannian manifold of SPD matrices, well-known machine learning techniques are not suited to classify these mat…
Transformer representation learning is necessary for dynamic multi-modal physiological data on small-cohort patients
Nanjing University（南京大学）；The Affiliated Drum Tower Hospital of Nanjing University Medical School（南京大学医学院附属鼓楼医院）；National Laboratory of Solid State Microstructure（固体微结构物理国家重点实验室）
本文提出结合 Transformer 表征与传统机器学习的 POD 预测框架，利用多模态生理数据构建首个两类患者数据集，证明多模态 Transformer 表征可提升术后 1–3 天谵妄诊断性能。
Toward Foundational Model for Sleep Analysis Using a Multimodal Hybrid Self-Supervised Learning Framework
Department of Brain and Cognitive Engineering, Korea University（脑科学与认知工程系，韩国大学）；Interdisciplinary Program in Precision Public Health, Korea University（精准公共卫生跨学科项目，韩国大学）；Department of Radiology, Stanford University School of Medicine（放射科，斯坦福大学医学院）；VA Palo Alto Health Care System（帕洛阿尔托医疗系统）；Department of Neurology, Korea University College of Medicine（神经病学系，韩国大学医学院）
Physiological signals that provide the objective repression of human affective states are attracted increasing attention in the emotion recognition field. However, the single signal is difficult to obtain completely and accurately description for emotion. Multiple physiological signals fusing models, building the uniform classification model by means of consistent and complementary information from different emotions to improve recognition performance. Original fusing models usually choose the particular classification method to recognition, which is ignoring different distribution of multiple…
University of California Irvine（加州大学尔湾分校）；University of California, Irvine（加州大学尔湾分校）
Monitoring and understanding affective states are important aspects of healthy functioning and treatment of mood-based disorders. Recent advancements of ubiquitous wearable technologies have increased the reliability of such tools in detecting and accurately estimating mental states (e.g., mood, stress, etc.), offering comprehensive and continuous monitoring of individuals over time. Previous attempts to model an individual's mental state were limited to subjective approaches or the inclusion of only a few modalities (i.e., phone, watch). Thus, the goal of our study was to investigate the capa…
As the result of the growing importance of the Human Computer Interface system, understanding human's emotion states has become a consequential ability for the computer. This paper aims to improve the performance of emotion recognition by conducting the complexity analysis of physiological signals. Based on AMIGOS dataset, we extracted several entropy-domain features such as Refined Composite Multi-Scale Entropy (RCMSE), Refined Composite Multi-Scale Permutation Entropy (RCMPE) from ECG and GSR signals, and Multivariate Multi-Scale Entropy (MMSE), Multivariate Multi-Scale Permutation Entropy (…
Sleep staging is fundamental for sleep assessment and disease diagnosis. Although previous attempts to classify sleep stages have achieved high classification performance, several challenges remain open: 1) How to effectively extract salient waves in multimodal sleep data; 2) How to capture the multi-scale transition rules among sleep stages; 3) How to adaptively seize the key role of specific modality for sleep staging. To address these challenges, we propose SalientSleepNet, a multimodal salient wave detection network for sleep staging. Specifically, SalientSleepNet is a temporal fully convo…
AnySleep: a channel-agnostic deep learning system for high-resolution sleep staging in multi-center cohorts
Department of Medical Engineering and Technomathematics, FH Aachen University of Applied Sciences（医学工程与技术学系，亚琛应用科学大学）；Department of Information and Computing Sciences, Utrecht University（信息与计算科学系，乌得勒支大学）；Institute for Data-Driven Technologies, FH Aachen University of Applied Sciences（数据驱动技术研究所，亚琛应用科学大学）
University of Massachusetts Amherst（马萨诸塞大学阿默斯特分校）；National University of Singapore（新加坡国立大学）；Singapore Management University（新加坡管理大学）
The availability of commercial wearable trackers equipped with features to monitor sleep duration and quality has enabled more useful sleep health monitoring applications and analyses. However, much research has reported the challenge of long-term user retention in sleep monitoring through these modalities. Since modern Internet users own multiple mobile devices, our work explores the possibility of employing ubiquitous mobile devices and passive WiFi sensing techniques to predict sleep duration as the fundamental measure for complementing long-term sleep monitoring initiatives. In this paper,…
