arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2603.09307cs.SD
收藏

日语共情口语对话中的话语情感意识验证时机检测

Paralinguistic Emotion-Aware Validation Timing Detection in Japanese Empathetic Spoken Dialogue

Zi Haur Pang, Yahui Fu, Yuan Gao, Tatsuya Kawahara

AI总结:

本文提出了一种结合语音和情感信息的模型,用于检测日语共情口语对话中的验证时机,通过自监督学习和多任务训练提升验证时机检测的准确性。

摘要:

情感验证是一种心理治疗沟通技术,涉及识别、理解和明确认可他人的感受和行为,这能加强联盟并减少负面情绪。为了最大化通过验证提供的情感支持,必须以适当的时间和频率进行传达。本研究从语音角度调查验证时机的检测。利用语音和情感信息,我们提出了一种具有语音和情感意识的模型,用于验证时机的检测,而无需依赖文本上下文。具体来说,我们首先在不同的HuBERT backbone上进行持续的自监督训练和微调,以获得(i) 一种语音意识的自监督学习(SSL)编码器和(ii) 多任务语音情感分类编码器。然后将这些编码器融合,并进一步在下游验证时机检测任务上微调联合模型。在TUT情感故事语料库(TESC)上的实验评估比较了多个模型、融合机制和训练策略,并证明所提出的方法在传统语音基线之上取得了显著改进。我们的结果表明,非语言的语音线索,当与情绪相关表示结合时,具有足够的信号来决定何时表达验证,提供了一条以语音为基础的路径,以实现更 empathetic 的人机交互。

英文摘要:

Emotional Validation is a psychotherapy communication technique that involves recognizing, understanding, and explicitly acknowledging another person's feelings and actions, which strengthens alliance and reduces negative affect. To maximize the emotional support provided by validation, it is crucial to deliver it with appropriate timing and frequency. This study investigates validation timing detection from the speech perspective. Leveraging both paralinguistic and emotional information, we propose a paralinguistic- and emotion-aware model for validation timing detection without relying on textual context. Specifically, we first conduct continued self-supervised training and fine-tuning on different HuBERT backbones to obtain (i) a paralinguistics-aware Self-Supervised Learning (SSL) encoder and (ii) a multi-task speech emotion classification encoder. We then fuse these encoders and further fine-tune the combined model on the downstream validation timing detection task. Experimental evaluations on the TUT Emotional Storytelling Corpus (TESC) compare multiple models, fusion mechanisms, and training strategies, and demonstrate that the proposed approach achieves significant improvements over conventional speech baselines. Our results indicate that non-linguistic speech cues, when integrated with affect-related representations, carry sufficient signal to decide when validation should be expressed, offering a speech-first pathway toward more empathetic human-robot interaction.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑