发表机构
Guangdong University of Technology; Hong Kong Metropolitan University（广东工业大学; 香港都会大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
本文提出RASG框架，通过角色感知视觉迁移和刺激引导推理，仅从听者视频和说话者音频预测听者情绪，在MER-Cross上达76.25%准确率，提升基线超17%，排名第二。
AI 中文摘要
本文提出了一种角色感知的刺激引导（RASG）框架用于对话者情绪识别，该框架仅从听者视频和说话者音频中预测听者情绪。RASG由角色感知的视觉迁移（RVT）和刺激引导的边界推理（SBR）模块组成，分别解决了因缺乏标注听者数据而导致的监督不匹配问题，以及视觉上相似的听者反应因依赖说话者上下文而具有歧义的问题。具体而言，RVT选择面部表情支持其情绪标签的说话者样本，然后过滤听者轨迹，并使用可靠的伪标签训练一个以听者为中心的视觉专家。SBR仅在视觉模型不确定时使用一个二分类语言推理器，它将说话者音频和文本视为上下文而非直接的情绪证据，以区分相似的听者反应。在MER-Cross数据集上进行的实验表明，RASG在MER-Cross上达到了76.25%的准确率，并将基线的性能提升了超过17%。我们的团队在ACM MM 2026的MER大挑战赛Track 1（MER-Cross）中排名第二。
英文摘要
In this paper, we propose a Role-aware Stimulus-guided (RASG) framework for interlocutor emotion recognition, which predicts listener emotions from listener-only videos and speaker-only audios. RASG consists of Role-aware Visual Transfer (RVT) and Stimulus-guided Boundary Reasoning (SBR) modules, which address supervision mismatch due to the lack of labeled listener data and ambiguity among visually similar listener reactions whose interpretation depends on speaker context, respectively. More specifically, RVT selects speaker samples whose facial expressions support their emotion labels. It then filters listener tracks and uses reliable pseudo-labels to train a listener-centric visual expert. SBR uses a two-class language reasoner only when the visual model is uncertain. It treats speaker audio and text as context rather than direct emotion evidence to distinguish similar listener reactions. Experiments conducted on MER-Cross dataset shows that RASG achieves 76.25\% on MER-Cross and improves the performance of the baseline over 17\%. Our team ranks second in Track 1 (MER-Cross) of the MER Grand Challenge at ACM MM 2026.
CommentsTechnical report of the second-place solution in Track 1 (MER-Cross) of the MER Grand Challenge at ACM MM 2026
