arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2402.13276eess.AScs.AIcs.SD
收藏

当LLM遇见声学标志:一种将语音集成到大语言模型中以用于抑郁症检测的高效方法

When LLMs Meets Acoustic Landmarks: An Efficient Approach to Integrate Speech into Large Language Models for Depression Detection

The University of New South Wales(新南威尔士大学);Nanyang Technological University(南洋理工大学);The Hong Kong Polytechnic University(香港理工大学)

Xiangyu Zhang, Hexin Liu, Kaishuai Xu, Qiquan Zhang, Daijiao Liu, Beena Ahmed, Julien Epps

AI总结:

本文提出一种利用声学标志将语音信号高效集成到大语言模型(LLMs)中的多模态方法,在DAIC-WOZ数据集上取得SOTA抑郁症检测效果,并拓展了LLMs处理语音信号的能力。

摘要:

抑郁症是全球心理健康中的一个关键问题,促使人们对基于人工智能的检测方法进行了广泛的研究。在各种人工智能技术中,大语言模型(LLMs)因其在心理健康护理应用中的多功能性而脱颖而出。然而,它们的主要局限在于其完全依赖于文本输入,这限制了它们的整体能力。此外,利用LLMs来识别和分析抑郁状态的研究仍然相对未被充分开发。在本文中,我们提出了一种创新方法,将声学语音信息整合到LLMs框架中,用于多模态抑郁症检测。我们研究了一种高效的抑郁症检测方法,该方法利用声学标志将语音信号整合到LLMs中。通过结合特定于口语单词发音的声学标志,我们的方法为文本转录添加了关键维度。这种整合还提供了对个体独特语音模式的洞察,揭示了个体潜在的心理状态。在DAIC-WOZ数据集上对所提出的方法进行评估,与现有的音频-文本基线相比,结果显示出了最先进的性能。此外,这种方法不仅对抑郁症的检测很有价值,而且代表了增强LLMs理解和处理语音信号能力的一个新视角。

英文摘要:

Depression is a critical concern in global mental health, prompting extensive research into AI-based detection methods. Among various AI technologies, Large Language Models (LLMs) stand out for their versatility in mental healthcare applications. However, their primary limitation arises from their exclusive dependence on textual input, which constrains their overall capabilities. Furthermore, the utilization of LLMs in identifying and analyzing depressive states is still relatively untapped. In this paper, we present an innovative approach to integrating acoustic speech information into the LLMs framework for multimodal depression detection. We investigate an efficient method for depression detection by integrating speech signals into LLMs utilizing Acoustic Landmarks. By incorporating acoustic landmarks, which are specific to the pronunciation of spoken words, our method adds critical dimensions to text transcripts. This integration also provides insights into the unique speech patterns of individuals, revealing the potential mental states of individuals. Evaluations of the proposed approach on the DAIC-WOZ dataset reveal state-of-the-art results when compared with existing Audio-Text baselines. In addition, this approach is not only valuable for the detection of depression but also represents a new perspective in enhancing the ability of LLMs to comprehend and process speech signals.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑