arXiv 2412.186142024-12-09eess.AScs.AIcs.CL

研究声学-文本情感不一致信息用于抑郁症自动检测

Investigating Acoustic-Textual Emotional Inconsistency Information for Automatic Depression Detection

Guangdong-Hong Kong-Macao Joint Laboratory of Human-Machine Intelligence-Synergy Systems, Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究院粤港澳人机协同智能系统联合实验室）；University of Chinese Academy of Sciences（中国科学院大学）；Good Mood Health Industry Group Co., Ltd（好心情健康产业集团有限公司）；Shanghai Jiao Tong University（上海交通大学）

Rongfeng Su, Changqing Xu, Xinyi Wu, Feng Xu, Xie Chen, Lan Wangt, Nan Yan

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文首次提出利用声学-文本情感不一致信息进行抑郁症检测，通过多模态交叉注意力捕捉ATEI特征，并结合Transformer模型与缩放融合策略提升不同严重程度患者的识别效果。

摘要：

以往研究表明，来自单一声学情感标签的情感特征能够提升抑郁症诊断的准确性。此外，根据情绪情境不敏感理论以及我们的初步研究，抑郁症个体可能会以出乎意料平静的方式传达负面情感内容，在自然对话中表现出高度的情感表达不一致性。迄今为止，很少有研究认识到并利用这种情感表达不一致性来进行抑郁症检测。本文提出了一种多模态交叉注意力方法，用于捕捉声学-文本情感不一致（ATEI）信息。该方法通过分析声学和文本领域中情感表达的复杂局部和长期依赖关系，以及两个领域内情感内容之间的不匹配来实现。随后，提出了一种基于Transformer的模型，将该ATEI信息与多种融合策略相结合以检测抑郁症。此外，采用了一种缩放技术在融合过程中调整ATEI特征的程度，从而增强模型在不同严重程度下识别抑郁症患者的能力。据我们所知，这项工作是首次将情感表达不一致信息纳入抑郁症检测的研究。在心理咨询对话数据集上的实验结果证明了我们方法的有效性。

英文摘要：

Previous studies have demonstrated that emotional features from a single acoustic sentiment label can enhance depression diagnosis accuracy. Additionally, according to the Emotion Context-Insensitivity theory and our pilot study, individuals with depression might convey negative emotional content in an unexpectedly calm manner, showing a high degree of inconsistency in emotional expressions during natural conversations. So far, few studies have recognized and leveraged the emotional expression inconsistency for depression detection. In this paper, a multimodal cross-attention method is presented to capture the Acoustic-Textual Emotional Inconsistency (ATEI) information. This is achieved by analyzing the intricate local and long-term dependencies of emotional expressions across acoustic and textual domains, as well as the mismatch between the emotional content within both domains. A Transformer-based model is then proposed to integrate this ATEI information with various fusion strategies for detecting depression. Furthermore, a scaling technique is employed to adjust the ATEI feature degree during the fusion process, thereby enhancing the model's ability to discern patients with depression across varying levels of severity. To best of our knowledge, this work is the first to incorporate emotional expression inconsistency information into depression detection. Experimental results on a counseling conversational dataset illustrate the effectiveness of our method.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。