Attention mechanisms are widely used to dramatically improve deep learning model performance in various fields. However, their general ability to improve the performance of physiological signal deep learning model is immature. In this study, we experimentally analyze four attention mechanisms (e.g., squeeze-and-excitation, non-local, convolutional block attention module, and multi-head self-attention) and three convolutional neural network (CNN) architectures (e.g., VGG, ResNet, and Inception) for two representative physiological signal prediction tasks: the classification for predicting hypotension and the regression for predicting cardiac output (CO). We evaluated multiple combinations for performance and convergence of physiological signal deep learning model. Accordingly, the CNN models with the spatial attention mechanism showed the best performance in the classification problem, whereas the channel attention mechanism achieved the lowest error in the regression problem. Moreover, the performance and convergence of the CNN models with attention mechanisms were better than stand-alone self-attention models in both problems. Hence, we verified that convolutional operation and attention mechanisms are complementary and provide faster convergence time, despite the stand-alone self-attention models requiring fewer parameters.
ASLSL: Adaptive shared latent structure learning with incomplete multi-modal physiological data for multi-dimensional emotional feature selection
School of Information Science and Technology, Beijing University of Technology, Beijing 100124, China（信息科学与技术学院，北京理工大学，北京）；School of Artificial Intelligence, Anhui University, Beijing 100124, China（人工智能学院，安徽大学，北京）
Toward Foundation Model for Multivariate Wearable Sensing of Physiological Signals
Halıcıoğlu Data Science Institute, University of California San Diego（哈利奇奥格鲁数据科学研究所，加州大学圣地亚哥分校）；Schools of Engineering, Arizona State University（工程学院，亚利桑那州立大学）
Integrating Biological and Machine Intelligence: Attention Mechanisms in Brain-Computer Interfaces
The School of Biomedical Engineering（生物医学工程学院）；The Guangdong Provincial Key Laboratory of Biomedical Measurements（生物医学测量与超声成像广东省重点实验室）；Shien-Ming Wu School of Intelligent Engineering（沈铭武智能工程学院）；Institute for Super Robotics（超机器人研究所）
Irregular sampling occurs in many time series modeling applications where it presents a significant challenge to standard deep learning models. This work is motivated by the analysis of physiological time series data in electronic health records, which are sparse, irregularly sampled, and multivariate. In this paper, we propose a new deep learning framework for this setting that we call Multi-Time Attention Networks. Multi-Time Attention Networks learn an embedding of continuous-time values and use an attention mechanism to produce a fixed-length representation of a time series containing a va…
QualityFM: a Multimodal Physiological Signal Foundation Model with Self-Distillation for Signal Quality Challenges in Critically Ill Patients
Department of Electronics, Information and Bioengineering, Politecnico di Milano（电子、信息与生物工程系，米兰理工学院）；State Key Laboratory of Industrial Control Technology, Zhejiang University（工业控制技术国家重点实验室，浙江大学）
In order to exploit representations of time-series signals, such as physiological signals, it is essential that these representations capture relevant information from the whole signal. In this work, we propose to use a Transformer-based model to process electrocardiograms (ECG) for emotion recognition. Attention mechanisms of the Transformer can be used to build contextualized representations for a signal, giving more importance to relevant parts. These representations may then be processed with a fully-connected network to predict emotions. To overcome the relatively small size of datasets w…
HeLo: Heterogeneous Multi-Modal Fusion with Label Correlation for Emotion Distribution Learning
College of Artificial Intelligence（人工智能学院）；Nanjing University of Aeronautics and Astronautics（南京航空航天大学）；School of Computer Science and Technology（计算机科学与技术学院）；Harbin Institute of Technology（哈尔滨工业大学）；Harbin Institute of Technology, Shenzhen（哈尔滨工业大学深圳分校）
NAP: Attention-Based Late Fusion for Automatic Sleep Staging
Faculty of Informatics, Università della Svizzera Italiana（信息学院，瑞士意大利大学）；Inst. of Digital Tech. for Personalized Healthcare, SUPSI（个性化医疗数字技术研究所，SUPSI）；Sleep Wake Epilepsy Center, Department of Neurology, Inselspital University of Bern（睡眠觉醒癫痫中心，神经科，伯恩大学医院）；Faculty of Informatics Università della Svizzera Italiana（信息学院，瑞士意大利大学）
Heart rate and respiratory rate measurement is a vital step for diagnosing many diseases. Non-contact camera based physiological measurement is more accessible and convenient in Telehealth nowadays than contact instruments such as fingertip oximeters since non-contact methods reduce risk of infection. However, remote physiological signal measurement is challenging due to environment illumination variations, head motion, facial expression, etc. It's also desirable to have a unified network which could estimate both heart rate and respiratory rate to reduce system complexity and latency. We prop…
Physiological signals that provide the objective repression of human affective states are attracted increasing attention in the emotion recognition field. However, the single signal is difficult to obtain completely and accurately description for emotion. Multiple physiological signals fusing models, building the uniform classification model by means of consistent and complementary information from different emotions to improve recognition performance. Original fusing models usually choose the particular classification method to recognition, which is ignoring different distribution of multiple…
University of Washington（华盛顿大学）；Microsoft Research（微软研究院）
Hundreds of millions of surgical procedures take place annually across the world, which generate a prevalent type of electronic health record (EHR) data comprising time series physiological signals. Here, we present a transferable embedding method (i.e., a method to transform time series signals into input features for predictive machine learning models) named PHASE (PHysiologicAl Signal Embeddings) that enables us to more accurately forecast adverse surgical outcomes based on physiological signals. We evaluate PHASE on minute-by-minute EHR data of more than 50,000 surgeries from two operating…
Queen Mary University of London（伦敦玛丽女王大学）；University of Genoa（热那亚大学）
Auditory attention to natural speech is a complex brain process. Its quantification from physiological signals can be valuable to improving and widening the range of applications of current brain-computer-interface systems, however it remains a challenging task. In this article, we present a dataset of physiological signals collected from an experiment on auditory attention to natural speech. In this experiment, auditory stimuli consisting of reproductions of English sentences in different auditory conditions were presented to 25 non-native participants, who were asked to transcribe the senten…
To investigate whether a deep learning model can detect Covid-19 from disruptions in the human body's physiological (heart rate) and rest-activity rhythms (rhythmic dysregulation) caused by the SARS-CoV-2 virus. We propose CovidRhythm, a novel Gated Recurrent Unit (GRU) Network with Multi-Head Self-Attention (MHSA) that combines sensor and rhythmic features extracted from heart rate and activity (steps) data gathered passively using consumer-grade smart wearable to predict Covid-19. A total of 39 features were extracted (standard deviation, mean, min/max/avg length of sedentary and active bout…
Efficient Pain Recognition via Respiration Signals: A Single Cross-Attention Transformer Multi-Window Fusion Pipeline
Foundation for Research \& Technology-Hellas Heraklion Greece；Foundation for Research \& Technology-Hellas；Hellenic Mediterranean University Heraklion Greece；Hellenic Mediterranean University
Multimodal wearable physiological data in daily life have been used to estimate self-reported stress labels. However, missing data modalities in data collection makes it challenging to leverage all the collected samples. Besides, heterogeneous sensor data and labels among individuals add challenges in building robust stress detection models. In this paper, we proposed a modality fusion network (MFN) to train models and infer self-reported binary stress labels under both complete and incomplete modality conditions. In addition, we applied personalized attention (PA) strategy to leverage persona…
Objective: To characterize the irregularity of the spectrum of a signal, spectral entropy is a widely adopted measure. However, such a metric is invariant under any permutation of the estimations of the powers of individual frequency components on a predefined grid. This erases the order structure inherent in the spectrum which is also an important aspect of irregularity of the signal. To disentangle the order structure and extract meaningful information from raw digital signal, novel analysis method is necessary. Approach: A novel method to depict the order structure by simply ranking power e…
