arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2404.14463cs.CLcs.AIcs.CYcs.LG
收藏

DAIC-WOZ:论在临床访谈自动抑郁检测中使用治疗师提示语的有效性

DAIC-WOZ: On the Validity of Using the Therapist's prompts in Automatic Depression Detection from Clinical Interviews

Idiap Research Institute(伊迪亚普研究所);Mathematics Research Center (CIMAT)(数学研究中心(CIMAT));Consejo Nacional de Humanidades, Ciencias y Tecnologías (CONAHCYT)(国家人文科学与技术委员会(CONAHCYT));Brno University of Technology(布尔诺理工大学)

Sergio Burdisso, Ernesto Reyes-Ramírez, Esaú Villatoro-Tello, Fernando Sánchez-Vega, Pastor López-Monroy, Petr Motlicek

AI总结:

本文针对DAIC-WOZ抑郁检测任务,揭示访谈者提示语带来的性能提升源于数据偏差而非模型方法,并通过消融与定性分析证明模型依赖针对性问题作为捷径。

摘要:

近年来,从对话数据中自动检测抑郁受到了广泛关注。DAIC-WOZ数据集由人类控制的虚拟代理进行访谈,已被广泛用于这一任务。近期研究报告称,将访谈者的提示语纳入模型后性能有所提升。在本研究中,我们假设这种提升可能主要源于这些提示语中存在的偏差,而非所提出的架构和方法。通过消融实验和定性分析,我们发现使用访谈者提示语的模型学会聚焦于访谈中询问过往心理健康问题经历的特定区域,并将其作为检测抑郁参与者的判别捷径。相比之下,使用参与者回答的模型会从整个访谈中收集证据。最后,为凸显该偏差的严重程度,我们通过有意利用它取得了0.90的F1分数,这是迄今为止在该数据集上仅使用文本信息所报告的最佳结果。我们的发现强调,在将访谈者提示语纳入模型时需保持谨慎,因为模型可能会无意中学会利用针对性提示,而不是学习刻画真正反映患者心理健康状况的语言和行为。

英文摘要:

Automatic depression detection from conversational data has gained significant interest in recent years. The DAIC-WOZ dataset, interviews conducted by a human-controlled virtual agent, has been widely used for this task. Recent studies have reported enhanced performance when incorporating interviewer's prompts into the model. In this work, we hypothesize that this improvement might be mainly due to a bias present in these prompts, rather than the proposed architectures and methods. Through ablation experiments and qualitative analysis, we discover that models using interviewer's prompts learn to focus on a specific region of the interviews, where questions about past experiences with mental health issues are asked, and use them as discriminative shortcuts to detect depressed participants. In contrast, models using participant responses gather evidence from across the entire interview. Finally, to highlight the magnitude of this bias, we achieve a 0.90 F1 score by intentionally exploiting it, the highest result reported to date on this dataset using only textual information. Our findings underline the need for caution when incorporating interviewers' prompts into models, as they may inadvertently learn to exploit targeted prompts, rather than learning to characterize the language and behavior that are genuinely indicative of the patient's mental health condition.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑