多模态隐马尔可夫模型用于持久情绪状态跟踪
Multimodal Hidden Markov Models for Persistent Emotional State Tracking
AI总结：
本文提出基于多模态valence- arousal表示的轻量框架，利用sticky factorial HDP-HMM实现对话情绪的持续状态跟踪，通过LLM-as-a-Judge等指标验证其比基线Gaussian HMM更高效且可解释。
摘要：
通过将单个话语的语气处理为整体情感来跟踪可解释的情绪弧线，对于应用，尤其是临床对话情境中的理解和引导沟通至关重要。现有情绪识别方法在话语层面操作，掩盖了真实对话动态中的持续阶段。我们提出一个轻量框架，利用多模态valence- arousal表示，通过sticky factorial HDP-HMM建模对话情绪为一系列潜在情绪状态。通过LLM-as-a-Judge、几何和时间一致性度量评估状态预测质量，证明sticky HDP-HMM在计算成本仅为LLM对话状态跟踪方法的分数时，产生比基线Gaussian HMM更可解释的状态序列。此外，在临床数据集中进行的问答实验表明，有意义的情绪阶段可以从多模态valence-arousal轨迹中可靠恢复，并用于通过上下文增强改善不稳定情绪状态下的LLM响应质量。此框架因此为大规模可解释、轻量且可操作的对话情绪动态分析开辟了新路径。
英文摘要：
Tracking an interpretable emotional arc of a conversation via the sentiment of individual utterances processed as a whole is central to both understanding and guiding communication in applied, especially clinical, conversational contexts. Existing approaches to emotion recognition operate at the utterance level, obscuring the persistent phases that characterize real conversational dynamics. We propose a lightweight framework that models conversational emotion as a sequence of latent emotional regimes using sticky factorial HDP-HMMs over multimodal valence-arousal representations derived from simultaneous video, audio and textual input. We evaluate the quality of regime prediction using LLM-as-a-Judge, geometric, and temporal consistency metrics, demonstrating that the sticky HDP-HMM produces more interpretable regime sequences than the baseline Gaussian HMM at a fraction of the computational cost of LLM-based dialogue state tracking methods. In addition, Question-Answer experiments in a clinical dataset suggest that meaningful emotional phases can reliably be recovered from multimodal valence-arousal trajectories and used to improve the quality of LLM responses in unstable affective regimes via context augmentation. This framework thus opens a path toward interpretable, lightweight, and actionable analysis of conversational emotion dynamics at scale.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
