TAG:基于动作单元的面部表情识别中的思维
TAG: Thinking with Action Unit Grounding for Facial Expression Recognition
AI总结：
TAG通过基于动作单元的视觉-语言框架，提升面部表情识别的推理可信度和视觉一致性。
摘要：
面部表情识别（FER）是一个细粒度的视觉理解任务，其中可靠的预测需要对局部化且有意义的面部线索进行推理。最近的视觉-语言模型（VLMs）能够为FER提供自然语言解释，但其推理通常是不基于现实的，产生流畅但难以验证的推理过程，这些推理与视觉证据联系较弱，容易产生幻觉，导致在不同数据集上表现不佳。我们提出了TAG（基于动作单元的思维），一个视觉-语言框架，该框架明确地将多模态推理限制在与面部动作单元（AUs）相关的面部区域上。TAG要求中间推理步骤必须基于与AUs相关的面部区域，从而产生带有可验证视觉证据的预测。该模型通过在基于AUs的推理轨迹上进行监督微调，再结合基于AUs的奖励进行强化学习进行训练。在RAF-DB、FERPlus和AffectNet上评估，TAG在超越强大的开源和闭源VLM基线的同时，同时提高了视觉可信度。消融和偏好研究进一步表明，基于AUs的奖励稳定了推理并缓解了幻觉，证明了结构化基于现实的中间表示对于在FER中实现可信的多模态推理的重要性。代码将在https://github.com/would1920/FER_TAG上提供。
英文摘要：
Facial Expression Recognition (FER) is a fine-grained visual understanding task where reliable predictions require reasoning over localized and meaningful facial cues. Recent vision--language models (VLMs) enable natural language explanations for FER, but their reasoning is often ungrounded, producing fluent yet unverifiable rationales that are weakly tied to visual evidence and prone to hallucination, leading to poor robustness across different datasets. We propose TAG (Thinking with Action Unit Grounding), a vision--language framework that explicitly constrains multimodal reasoning to be supported by facial Action Units (AUs). TAG requires intermediate reasoning steps to be grounded in AU-related facial regions, yielding predictions accompanied by verifiable visual evidence. The model is trained via supervised fine-tuning on AU-grounded reasoning traces followed by reinforcement learning with an AU-aware reward that aligns predicted regions with external AU detectors. Evaluated on RAF-DB, FERPlus, and AffectNet, TAG consistently outperforms strong open-source and closed-source VLM baselines while simultaneously improving visual faithfulness. Ablation and preference studies further show that AU-grounded rewards stabilize reasoning and mitigate hallucination, demonstrating the importance of structured grounded intermediate representations for trustworthy multimodal reasoning in FER. The code will be available at https://github.com/would1920/FER_TAG .
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
