专家混合模型用于从访谈和阅读任务中识别抑郁症
Mixture of Experts for Recognizing Depression from Interview and Reading Tasks
AI总结：
本文提出利用多模态融合和专家混合模型，从访谈和阅读任务的语音中识别抑郁症，实现87%的准确率和86.66%的F1分数。
摘要：
抑郁症是一种心理障碍，可能导致心理、生理和社会症状。语音已被证明是早期识别抑郁症的客观标志。然而，现有方法依赖于自发语音，忽视了阅读语音的信息，使用转录文本往往难以获得或具有高词错误率，并未关注输入条件计算方法。为此，本文首次在抑郁症识别任务中获取自发和阅读语音的表示，利用多模态融合方法，并在单一深度神经网络中使用专家混合（MoE）模型。具体来说，我们使用对应于访谈和阅读任务的音频文件，并将每个音频文件转换为log-Mel频谱图、delta和delta-delta。然后，两种任务的图像表示通过共享的AlexNet模型进行处理。AlexNet模型的输出作为输入传递给多模态融合方法。得到的向量通过MoE模块进行处理。本文采用了三种MoE变体，即稀疏门控MoE和基于分解的多线性MoE。研究结果表明，所提出的方法在Androids语料库上实现了87.00%的准确率和86.66%的F1分数。
英文摘要：
Depression is a mental disorder and can cause a variety of symptoms, including psychological, physical, and social. Speech has been proved an objective marker for the early recognition of depression. For this reason, many studies have been developed aiming to recognize depression through speech. However, existing methods rely on the usage of only the spontaneous speech neglecting information obtained via read speech, use transcripts which are often difficult to obtain (manual) or come with high word-error rates (automatic), and do not focus on input-conditional computation methods. To resolve these limitations, this is the first study in depression recognition task obtaining representations of both spontaneous and read speech, utilizing multimodal fusion methods, and employing Mixture of Experts (MoE) models in a single deep neural network. Specifically, we use audio files corresponding to both interview and reading tasks and convert each audio file into log-Mel spectrogram, delta, and delta-delta. Next, the image representations of the two tasks pass through shared AlexNet models. The outputs of the AlexNet models are given as input to a multimodal fusion method. The resulting vector is passed through a MoE module. In this study, we employ three variants of MoE, namely sparsely-gated MoE and multilinear MoE based on factorization. Findings suggest that our proposed approach yields an Accuracy and F1-score of 87.00% and 86.66% respectively on the Androids corpus.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
