AI 中文总结
针对流视频情感理解的实时需求，提出TRACE框架，通过轨迹感知可靠性机制优化多模态推理分配，在StreamMER等数据集上提升了准确率与成本的权衡效果。
AI 中文摘要
视频情感理解通常被作为离线分类问题研究，即预测前可获取完整视频片段。然而实时交互需要基于不完整且不断演变的证据做出情感决策。本文将流视频情感理解作为对不断演变的情感信念的可靠性感知决策过程进行研究。在此场景下，当潜在信念轨迹不稳定或在情感类别间反复切换时，单一的高置信度前缀预测仍可能不可靠。我们提出TRACE（Trajectory-aware Reliability Framework，轨迹感知可靠性框架），该框架从流音频前缀形成低延迟情感信念，通过置信度、熵、稳定性和类别切换模式估计可靠性，并选择性调用视觉、文本及相邻话语证据进行上下文信念重新解释。TRACE将稳定案例保留在低延迟在线路径中，同时为仍存在歧义的不确定案例分配更强的多模态推理资源。在StreamMER、MELD和MER2024数据集上的实验表明，TRACE提升了准确率-成本权衡效果，在保留大部分全上下文增益的同时减少了不必要的上下文推理。
英文摘要
Video emotion understanding is commonly studied as an offline classification problem, where the complete video segment is available before prediction. Real-time interaction, however, requires emotion decisions from incomplete and evolving evidence. This paper studies streaming video emotion understanding as a reliability-aware decision process over evolving emotion beliefs. In this setting, a single confident prefix prediction can still be unreliable when the underlying belief trajectory is unstable or repeatedly switches across emotion classes. We propose TRACE, a trajectory-aware reliability framework that forms low-latency emotion beliefs from streaming audio prefixes, estimates reliability from confidence, entropy, stability, and class-switching patterns, and selectively invokes contextual belief reinterpretation with visual, textual, and neighboring-utterance evidence. TRACE keeps stable cases in the low-latency online pathway while allocating stronger multimodal reasoning to uncertain cases that remain ambiguous. Experiments on StreamMER, MELD, and MER2024 show that TRACE improves the accuracy-cost trade-off, retaining most full-context gains while reducing unnecessary contextual reasoning.
CommentsAccepted by EMNLP2026
