从粗粒度到细粒度：用于动态情绪识别的细粒度语言线索与视觉显著区域的跨模态对齐
From Coarse to Nuanced: Cross-Modal Alignment of Fine-Grained Linguistic Cues and Visual Salient Regions for Dynamic Emotion Recognition
AI总结：
提出GRACE方法，通过整合动态运动建模、语义文本细化与token级跨模态对齐，精确定位情绪时空特征，在三个基准数据集上取得SOTA性能。
摘要：
动态面部表情识别（DFER）旨在从随时间演化的面部动作中识别人类情绪，在情感计算中发挥着关键作用。尽管近期的视觉-语言方法引入了语义文本描述来指导表情识别，但现有方法仍面临两个关键局限：它们往往未充分利用生成文本中蕴含的细微情绪线索，且尚未引入足够有效的机制来过滤与情绪表达无关的面部动态。为填补这些空白，我们提出了GRACE，即用于跨模态情绪识别的粒度表示对齐方法。该方法整合了动态运动建模、语义文本细化以及token级别的跨模态对齐，以促进对情绪显著的时空特征进行精确定位。我们的方法通过粗粒度到细粒度的情感文本增强（CATE）模块构建情绪感知的文本描述，并通过运动差异加权机制突出与表情相关的面部运动。这些细化后的语义和视觉信号使用熵正则化最优传输在token级别进行对齐。在三个基准数据集上的实验表明，我们的方法显著提升了识别性能，特别是在具有模糊或类别不平衡情绪的挑战性环境中，并在UAR和WAR指标上均取得了新的最优（SOTA）结果。
英文摘要：
Dynamic Facial Expression Recognition (DFER) aims to identify human emotions from temporally evolving facial movements and plays a critical role in affective computing. While recent vision-language approaches have introduced semantic textual descriptions to guide expression recognition, existing methods still face two key limitations: they often underutilize the subtle emotional cues embedded in generated text, and they have yet to incorporate sufficiently effective mechanisms for filtering out facial dynamics that are irrelevant to emotional expression. To address these gaps, We propose GRACE, Granular Representation Alignment for Cross-modal Emotion recognition that integrates dynamic motion modeling, semantic text refinement, and token-level cross-modal alignment to facilitate the precise localization of emotionally salient spatiotemporal features. Our method constructs emotion-aware textual descriptions via a Coarse-to-fine Affective Text Enhancement (CATE) module and highlights expression-relevant facial motion through a motion-difference weighting mechanism. These refined semantic and visual signals are aligned at the token level using entropy-regularized optimal transport. Experiments on three benchmark datasets demonstrate that our method significantly improves recognition performance, particularly in challenging settings with ambiguous or imbalanced emotion classes, establishing new state-of-the-art (SOTA) results in terms of both UAR and WAR.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
