Employing Multimodal Machine Learning for Stress Detection
Symbiosis Institute of Technology, Symbiosis International (Deemed University);Symbiosis Centre for Applied Artificial Intelligence, Symbiosis International (Deemed University)
In the current age, human lifestyle has become more knowledge oriented leading to generation of sedentary employment. This has given rise to a number of health and mental disorders. Mental wellness is one of the most neglected but crucial aspects of today's world. Mental health issues can, both directly and indirectly, affect other sections of human physiology and impede an individual's day-to-day activities and performance. However, identifying the stress and finding the stress trend for an individual leading to serious mental ailments is challenging and involves multiple factors. Such identification can be achieved accurately by fusing these multiple modalities (due to various factors) arising from behavioral patterns. Certain techniques are identified in the literature for this purpose; however, very few machine learning-based methods are proposed for such multimodal fusion tasks. In this work, a multimodal AI-based framework is proposed to monitor a person's working behavior and stress levels. We propose a methodology for efficiently detecting stress due to workload by concatenating heterogeneous raw sensor data streams (e.g., face expressions, posture, heart rate, computer interaction). This data can be securely stored and analyzed to understand and discover personalized unique behavioral patterns leading to mental strain and fatigue. The contribution of this work is twofold; proposing a multimodal AI-based strategy for fusion to detect stress and its level and secondly identify a stress pattern over a period of time. We were able to achieve 96.09% accuracy on the test set in stress detection and classification. Further, we reduce the stress scale prediction model loss to 0.036 using these modalities. This work can prove important for the community at large, specifically those working sedentary jobs to monitor and identify stress levels, especially in current times of COVID-19.
Fusion of Physiological and Behavioural Signals on SPD Manifolds with Application to Stress and Pain Detection
CNRS(法国国家科学研究中心);University of Lille(里尔大学);Centrale Lille(里尔中央理工学院);IMT Nord Europe(法国北部高等矿业电信学院);Institut Mines-Télécom(法国矿业电信学院);INSERM(法国国家健康与医学研究院)
The MuSe 2022 Multimodal Sentiment Analysis Challenge: Humor, Emotional Reactions, and Stress
University of Augsburg(奥格斯堡大学);Hume AI(休谟人工智能);University of Passau(帕绍大学);Recoro;University of Ulm(乌尔姆大学);Nanyang Technological University(南洋理工大学);Imperial College London(伦敦帝国理工学院)
Tracing Human Stress from Physiological Signals using UWB Radar
Guangzhou University(广州大学);Guangxi University(广西大学);Fudan University(复旦大学);Huazhong University of Science and Technology(华中科技大学);Nanjing University of Aeronautics and Astronautics(南京航空航天大学);Singapore University of Technology and Design(新加坡科技设计大学)
Stress-Testing Multimodal Foundation Models for Crystallographic Reasoning
Dept. of Electrical and Computer Engineering, Texas A&M University(电气与计算机工程系,德克萨斯A&M大学);College of Science and Engineering, Hamad Bin Khalifa University(科学与工程学院,哈马德·本·卡西姆大学);Dept. of Electrical and Computer Engineering, Texas A&M University at Qatar(电气与计算机工程系,德克萨斯A&M大学(卡塔尔));Dept. of Prosthetics and Orthotics, Ankara University(假肢与矫形系,安卡拉大学)
Planning for Cooler Cities: A Multimodal AI Framework for Predicting and Mitigating Urban Heat Stress through Urban Landscape Transformation
Department of City and Regional Planning, University of Pennsylvania(城市与区域规划系,宾夕法尼亚大学);Guangdong Key Laboratory for Urban Informatics, Guangdong-Hong Kong-Macao Joint Laboratory for Smart Cities(广东城市信息关键实验室、粤港澳大湾区智慧城市联合实验室);Shenzhen Key Laboratory of Spatial Information Smart Sensing(深圳空间信息智能感知关键实验室);Department of Urban Informatics, School of Architecture and Urban Planning, Shenzhen University(城市信息系,深圳大学);College of Big Data and Internet, Shenzhen Technology University(大数据与互联网学院,深圳科技大学)
Real-Time Stress Detection via Photoplethysmogram Signals: Implementation of a Combined Continuous Wavelet Transform and Convolutional Neural Network on Resource-Constrained Microcontrollers
University of Tehran(德黑兰大学);Kettering University(凯特林大学)
Differential Private Federated Transfer Learning for Mental Health Monitoring in Everyday Settings: A Case Study on Stress Detection
University of California, Irvine(加利福尼亚大学欧文分校);School of Nursing, University of California, Irvine(加利福尼亚大学欧文分校护理学院);Institute for Future Health, University of California, Irvine(加利福尼亚大学欧文分校未来健康研究所)
Efficient Quantification of Multimodal Interaction at Sample Level
Gaoling School of Artificial Intelligence, Renmin University of China, Beijing, China(中国人民大学语言学院);Tencent Data Platform(腾讯数据平台);Tsinghua University, Beijing, China(清华大学);Beijing Key Laboratory of Research on Large Models(北京大型模型研究关键实验室);Engineering Research Center of Next-Generation Intelligent Search(下一代智能搜索工程研究中心)