arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2408.07313cs.HC
收藏

探索大语言模型评估基于EEG的心理健康多模态数据

Exploring Large-Scale Language Models to Evaluate EEG-Based Multimodal Data for Mental Health

Yongquan Hu, Shuning Zhang, Ting Dang, Hong Jia, Flora D. Salim, Wen Hu, Aaron J. Quigley

AI总结:

本研究针对心理健康评估中单模态研究的局限,采用零样本和少样本提示技术,用大语言模型处理含EEG的多模态数据,经3个数据集验证,多模态方法及1-shot学习效果更优。

摘要:

将脑电图(EEG)等生理信号与访谈音频等其他数据相结合,可为心理状态或神经障碍的研究提供有价值的多模态视角。大语言模型(LLMs)的最新进展使其有望成为心理健康评估领域的“健康智能体”。但当前研究主要聚焦单一数据模态,为通过多模态数据深化相关认知留下了空间。本研究旨在推进这一方向,探究将LLMs用于心理健康评估的多模态数据处理方法,具体采用零样本(zero-shot)和少样本(few-shot)提示技术。研究使用3个包含EEG、面部表情及音频(文本)的数据集开展抑郁分类与情绪分类任务。结果表明,在心理健康评估中,多模态信息相比单模态方法具备显著优势。值得注意的是,将EEG与音频、图像等LLM常用模态相结合展现出可观潜力。此外,研究发现1-shot学习相比零样本学习方法效果更优。

英文摘要:

Integrating physiological signals such as electroencephalogram (EEG), with other data such as interview audio, may offer valuable multimodal insights into psychological states or neurological disorders. Recent advancements with Large Language Models (LLMs) position them as prospective ``health agents'' for mental health assessment. However, current research predominantly focus on single data modalities, presenting an opportunity to advance understanding through multimodal data. Our study aims to advance this approach by investigating multimodal data using LLMs for mental health assessment, specifically through zero-shot and few-shot prompting. Three datasets are adopted for depression and emotion classifications incorporating EEG, facial expressions, and audio (text). The results indicate that multimodal information confers substantial advantages over single modality approaches in mental health assessment. Notably, integrating EEG alongside commonly used LLM modalities such as audio and images demonstrates promising potential. Moreover, our findings reveal that 1-shot learning offers greater benefits compared to zero-shot learning methods.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑