利用多模态机器学习中的嵌入技术进行精神疾病评估
Leveraging Embedding Techniques in Multimodal Machine Learning for Mental Illness Assessment
AI总结：
本文研究多模态机器学习在精神疾病评估中的应用，通过结合文本、音频和视频数据，采用CNN-BiLSTM架构、话语级分块及决策级融合（含LLM预测），在抑郁症和PTSD检测中分别达到94.8%和96.2%的平衡准确率，显著提升了诊断的客观性和可扩展性。
摘要：
抑郁症和PTSD等精神障碍在全球的日益流行，需要客观且可扩展的诊断工具。传统的临床评估在可及性、客观性和一致性方面往往面临局限。本文研究了多模态机器学习（MMML）应对这些挑战的潜力，利用文本、音频和视频数据中可用的互补信息。我们的方法涉及对各种数据预处理技术的全面分析，包括新颖的分块和基于话语的格式化策略。我们系统地评估了每种模态的一系列最先进的嵌入模型，并采用卷积神经网络（CNNs）和双向LSTM网络（BiLSTMs）进行特征提取。我们探索了数据级、特征级和决策级融合技术，包括对大型语言模型（LLM）预测的新颖整合。我们还研究了用支持向量机替换多层感知器分类器的影响。我们将分析扩展到使用PHQ-8和PCL-C分数进行严重程度预测以及多类分类（考虑共病情况）。我们的结果表明，基于话语的分块显著提高了性能，特别是对于文本和音频模态。结合LLM预测的决策级融合实现了最高准确率，抑郁症检测的平衡准确率为94.8%，PTSD检测为96.2%。CNN-BiLSTM架构与话语级分块相结合，加上外部LLM的整合，为精神健康状况的检测和评估提供了一种强大且细致的方法。我们的发现突出了MMML在开发更准确、可及和个性化的精神医疗工具方面的潜力。
英文摘要：
The increasing global prevalence of mental disorders, such as depression and PTSD, requires objective and scalable diagnostic tools. Traditional clinical assessments often face limitations in accessibility, objectivity, and consistency. This paper investigates the potential of multimodal machine learning to address these challenges, leveraging the complementary information available in text, audio, and video data. Our approach involves a comprehensive analysis of various data preprocessing techniques, including novel chunking and utterance-based formatting strategies. We systematically evaluate a range of state-of-the-art embedding models for each modality and employ Convolutional Neural Networks (CNNs) and Bidirectional LSTM Networks (BiLSTMs) for feature extraction. We explore data-level, feature-level, and decision-level fusion techniques, including a novel integration of Large Language Model (LLM) predictions. We also investigate the impact of replacing Multilayer Perceptron classifiers with Support Vector Machines. We extend our analysis to severity prediction using PHQ-8 and PCL-C scores and multi-class classification (considering co-occurring conditions). Our results demonstrate that utterance-based chunking significantly improves performance, particularly for text and audio modalities. Decision-level fusion, incorporating LLM predictions, achieves the highest accuracy, with a balanced accuracy of 94.8% for depression and 96.2% for PTSD detection. The combination of CNN-BiLSTM architectures with utterance-level chunking, coupled with the integration of external LLM, provides a powerful and nuanced approach to the detection and assessment of mental health conditions. Our findings highlight the potential of MMML for developing more accurate, accessible, and personalized mental healthcare tools.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
