arXiv 2504.218002025-09-20cs.CLcs.AIcs.CYcs.HC

合成治疗对话有多真实？评估延长暴露疗法对话中的保真度

How Real Are Synthetic Therapy Conversations? Evaluating Fidelity in Prolonged Exposure Dialogues

College of Information Sciences and Technology, Penn State University（宾夕法尼亚州立大学信息科学与技术学院）；Department of Psychiatry and Behavioral Sciences, Emory University（埃默里大学精神病学与行为科学系）；School of Interactive Computing, Georgia Tech（佐治亚理工学院交互计算学院）；School of Psychology, Georgia Tech（佐治亚理工学院心理学学院）

Suhas BN, Dominik Mattioli, Saeed Abdullah, Rosa I. Arriaga, Chris W. Wiese, Andrew M. Sherrill

备注：: 10 pages, 5 tables. Accepted for Poster presentation at EMNLP 2025

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文系统比较了PTSD延长暴露疗法的合成对话与真实对话，提出PE特异性保真度评估框架，发现合成数据能复现关键语言特征但在微妙治疗动态上仍有差距，为临床部署前基准测试提供了模型无关工具。

摘要：

医疗保健领域采用合成数据是出于隐私担忧、数据访问限制和高昂的标注成本。我们探索将用于创伤后应激障碍（PTSD）的合成延长暴露（Prolonged Exposure, PE）疗法对话作为训练临床模型的可扩展替代方案。我们使用语言、结构和方案特定指标（如话轮转换和治疗保真度）系统比较真实对话与合成对话。我们引入并评估了PE特定指标，为评估超越表面流畅度的临床保真度提供了一个新颖框架。我们的研究结果表明，虽然合成数据成功缓解了数据稀缺问题并保护了隐私，但捕捉最微妙的治疗动态仍然是一项复杂挑战。合成对话成功复现了真实对话的关键语言特征，例如取得了相似的可读性得分（89.2对比88.1），但在一些关键保真度标记（如痛苦监测）上表现出差异。这一比较凸显了对超越表面流畅度的保真度感知指标的需求，以识别具有临床意义的细微差别。我们的模型无关框架是开发者和临床医生在敏感应用中部署生成模型之前对其保真度进行基准测试的关键工具。我们的研究结果有助于阐明合成数据在哪些方面可以有效补充真实世界数据集，同时也指出了未来需要改进的领域。

英文摘要：

Synthetic data adoption in healthcare is driven by privacy concerns, data access limitations, and high annotation costs. We explore synthetic Prolonged Exposure (PE) therapy conversations for PTSD as a scalable alternative for training clinical models. We systematically compare real and synthetic dialogues using linguistic, structural, and protocol-specific metrics like turn-taking and treatment fidelity. We introduce and evaluate PE-specific metrics, offering a novel framework for assessing clinical fidelity beyond surface fluency. Our findings show that while synthetic data successfully mitigates data scarcity and protects privacy, capturing the most subtle therapeutic dynamics remains a complex challenge. Synthetic dialogues successfully replicate key linguistic features of real conversations, for instance, achieving a similar Readability Score (89.2 vs. 88.1), while showing differences in some key fidelity markers like distress monitoring. This comparison highlights the need for fidelity-aware metrics that go beyond surface fluency to identify clinically significant nuances. Our model-agnostic framework is a critical tool for developers and clinicians to benchmark generative model fidelity before deployment in sensitive applications. Our findings help clarify where synthetic data can effectively complement real-world datasets, while also identifying areas for future refinement.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。