arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新，休息一下吧，周末愉快~~
arXiv 2508.08093cs.CVcs.LGcs.MMeess.AS
收藏

MDD-Net：通过Mutual Transformer进行多模态抑郁症检测

MDD-Net: Multimodal Depression Detection through Mutual Transformer

Centre for Pattern Analysis and Machine Intelligence, Department of Electrical and Computer Engineering, University of Waterloo（模式分析与机器智能中心，电气与计算机工程系，滑铁卢大学）；School of Engineering and Computing, Department of Computer Science and Engineering, American University of Ras Al Khaimah（工程与计算学院，计算机科学与工程系，阿联酋拉线哈姆市美国大学）

Md Rezwanul Haque, Md. Milon Islam, S M Taslim Uddin Raju, Hamdi Altaheri, Lobna Nassar, Fakhri Karray

备注：
Accepted for the 2025 IEEE International Conference on Systems, Man, and Cybernetics (SMC), Vienna, Austria
阅读 PDF
arXiv 页面 ↗HTML 原文 ↗

AI总结：

本研究提出MDD-Net，利用mutual transformer融合声学和视觉特征进行抑郁症检测，在D-Vlog数据集上F1-Score较现有最优方法提升最高达17.37%。

摘要：

抑郁症是一种严重影响个人情绪和身体健康的主要心理健康疾病。从社交媒体平台收集数据的简便性引起了人们对正确利用这些信息进行心理健康研究的极大兴趣。本研究提出了一种多模态抑郁症检测网络（MDD-Net），利用从社交网络获取的声学和视觉数据，其中利用mutual transformers来高效提取和融合多模态特征，以实现高效的抑郁症检测。MDD-Net包含四个核心模块：用于检索相关声学属性的声学特征提取模块，用于提取重要高层模式的视觉特征提取模块，用于计算生成特征之间的相关性并融合来自多种模态特征的mutual transformer，以及使用融合后的特征表示检测抑郁症的检测层。使用多模态D-Vlog数据集进行了大量实验，研究结果表明，所开发的多模态抑郁症检测网络在F1-Score上最高超出现有最先进水平17.37%，证明了所提出系统的卓越性能。源代码可在https://github.com/rezwanh001/Multimodal-Depression-Detection获取。

英文摘要：

Depression is a major mental health condition that severely impacts the emotional and physical well-being of individuals. The simple nature of data collection from social media platforms has attracted significant interest in properly utilizing this information for mental health research. A Multimodal Depression Detection Network (MDD-Net), utilizing acoustic and visual data obtained from social media networks, is proposed in this work where mutual transformers are exploited to efficiently extract and fuse multimodal features for efficient depression detection. The MDD-Net consists of four core modules: an acoustic feature extraction module for retrieving relevant acoustic attributes, a visual feature extraction module for extracting significant high-level patterns, a mutual transformer for computing the correlations among the generated features and fusing these features from multiple modalities, and a detection layer for detecting depression using the fused feature representations. The extensive experiments are performed using the multimodal D-Vlog dataset, and the findings reveal that the developed multimodal depression detection network surpasses the state-of-the-art by up to 17.37% for F1-Score, demonstrating the greater performance of the proposed system. The source code is accessible at https://github.com/rezwanh001/Multimodal-Depression-Detection.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

↑