面向连续情感估值的阶段自适应可靠性建模
Stage-Adaptive Reliability Modeling for Continuous Valence-Arousal Estimation
AI总结:
本文提出SAGE框架,通过阶段自适应可靠性建模提升多模态环境下连续情感估值的稳定性与准确性。
摘要:
在真实环境中进行连续情感估值面临挑战,因为音频视觉信号的可靠性不一致且受交互依赖性变化影响。现有方法主要关注时间动态建模,往往忽略了模态可靠性在交互阶段间显著变化的事实。为解决这一问题,我们提出SAGE,一种阶段自适应可靠性建模框架,该框架在多模态整合过程中明确估计和校准模态的置信度。SAGE引入一种可靠性感知融合机制,根据阶段依赖的信息性动态重新平衡音频和视觉表示,防止不可靠信号主导预测过程。通过将可靠性估计与特征表示分离,所提出的框架在跨模态噪声、遮挡和变化的交互条件下实现了更稳定的emotion估计。在Aff-Wild2基准上的广泛实验表明,SAGE在一致性相关系数评分上 consistently 比现有多模态融合方法有所提升,突显了以可靠性驱动建模在连续情感预测中的有效性。
英文摘要:
Continuous valence-arousal estimation in real-world environments is challenging due to inconsistent modality reliability and interaction-dependent variability in audio-visual signals. Existing approaches primarily focus on modeling temporal dynamics, often overlooking the fact that modality reliability can vary substantially across interaction stages. To address this issue, we propose SAGE, a Stage-Adaptive reliability modeling framework that explicitly estimates and calibrates modality-wise confidence during multimodal integration. SAGE introduces a reliability-aware fusion mechanism that dynamically rebalances audio and visual representations according to their stage-dependent informativeness, preventing unreliable signals from dominating the prediction process. By separating reliability estimation from feature representation, the proposed framework enables more stable emotion estimation under cross-modal noise, occlusion, and varying interaction conditions. Extensive experiments on the Aff-Wild2 benchmark demonstrate that SAGE consistently improves concordance correlation coefficient scores compared with existing multimodal fusion approaches, highlighting the effectiveness of reliability-driven modeling for continuous affect prediction.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。