Exploring Large-Scale Language Models to Evaluate EEG-Based Multimodal Data for Mental Health
Yongquan Hu, Shuning Zhang, Ting Dang, Hong Jia, Flora D. Salim, Wen Hu, Aaron J. Quigley
备注:
6 pages; UbiComp Companion '24, Companion of the 2024 ACM International Joint Conference on Pervasive and Ubiquitous Computing, October 5--9, 2024}{Melbourne, VIC, Australia
Integrating physiological signals such as electroencephalogram (EEG), with other data such as interview audio, may offer valuable multimodal insights into psychological states or neurological disorders. Recent advancements with Large Language Models (LLMs) position them as prospective ``health agents'' for mental health assessment. However, current research predominantly focus on single data modalities, presenting an opportunity to advance understanding through multimodal data. Our study aims to advance this approach by investigating multimodal data using LLMs for mental health assessment, specifically through zero-shot and few-shot prompting. Three datasets are adopted for depression and emotion classifications incorporating EEG, facial expressions, and audio (text). The results indicate that multimodal information confers substantial advantages over single modality approaches in mental health assessment. Notably, integrating EEG alongside commonly used LLM modalities such as audio and images demonstrates promising potential. Moreover, our findings reveal that 1-shot learning offers greater benefits compared to zero-shot learning methods.
Monitoring mental workload by EEG during a game in Virtual Reality
在活动期间,了解用户的心理工作负荷(MWL)允许改善人机交互(HMI)。事实上,MWL对个人及其与环境的互动产生影响。因此,监测是一个至关重要的问题。在这种情况下,我们创建了基于N-back任务(通常用于测量MWL)的虚拟游戏Back to Pizza。在这个更有趣的变体中,用户必须执行比萨食品卡车的客户的订单。这是一个互动游戏,涉及IHM'23会议的观众,选择几个参数,如成分的数量。在此体验期间,目标是通过脑电图(EEG)实时的测量MWL,并向观众提供MWL水平的视觉反馈。通过这个演示,我们提出了一个虚拟互动游戏的概念,测量MWL在实时的。
Financial technologies (FinTech) for mental health: The potential of objective financial data to better understand the relationships between financial behavior and mental health
"I Don't Want My Mental Health App To Give Me Mental Health Barriers": Unpacking The Need For Digital Mental Health Tracking Services With And For The Blind Community
University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校)
Beyond Content Exposure: Systemic Factors Driving Moderators' Mental Health Crisis in Africa
University of Minnesota(明尼苏达大学);Technische Universität Berlin(柏林工业大学);Carnegie Mellon University(卡内基梅隆大学);Pan African Christian University(泛非基督大学);The Distributed AI Research Institute(分布式人工智能研究所);Boston University(波士顿大学)
Sensors and Systems for Monitoring Mental Fatigue: A systematic review
University of Massachusetts Boston(马萨诸塞大学波士顿分校);Australian Catholic University(澳大利亚天主教大学);Nepalese Army Institute of Health Sciences College of Medicine(尼泊尔陆军卫生科学学院医学院)
Modified TSception for Analyzing Driver Drowsiness and Mental Workload from EEG
Department of Computer Science and Engineering, Indian Institute of Technology, Roorkee, Uttarakhand, 247667, India(计算机科学与工程系,印度理工学院,罗尔基,乌塔兰德,印度);Department of Computer Science and Engineering, OP Jindal University, Raigarh, Chhattisgarh, 496109, India(计算机科学与工程系,OP金达尔大学,拉里加,恰特内格尔,印度);Department of Computer Science, Electrical and Space Engineering, Luleå Tekniska Universitet, Luleå, 97187, Sweden(计算机科学、电气与航天工程系,卢勒奥技术大学,卢勒奥,瑞典);Department of Computer Science and Engineering, Indian Institute of Technology, Dhanbad, Jharkhand, 826007, India(计算机科学与工程系,印度理工学院,丹巴德,贾坎德,印度)
Fine-Tuning Large Language Models Using EEG Microstate Features for Mental Workload Assessment
School of Computer Science, Technological University Dublin, Dublin, Ireland(计算机科学学院,都柏林技术大学,都柏林,爱尔兰);Artificial Intelligence and Cognitive Load Lab, Applied Intelligence Research Centre, School of Computer Science, Technological University Dublin, Dublin, Ireland(人工智能与认知负荷实验室,应用智能研究中心,计算机科学学院,都柏林技术大学,都柏林,爱尔兰)
A Checklist for Trustworthy, Safe, and User-Friendly Mental Health Chatbots
University of Illinois Urbana-Champaign, Urbana, United States(伊利诺伊大学厄巴纳-香槟分校);Indiana University Indianapolis, Indianapolis, United States(印第安纳大学印第安纳波利斯分校)
Inclusive Brains(包容大脑);Wavestone;Human Technology Foundation(人类科技基金会);Mohamed bin Zayed University of Artificial Intelligence(Mohamed bin Zayed人工智能大学);Institute for Artificial Intelligence, Biotech Dental Group(人工智能研究所,生物技术牙科集团);IBM Technology(IBM技术);IBM France Lab(IBM法国实验室)