arXiv 2605.198212026-07-13cs.CV

LaCoVL-FER: 一种结合视觉-语言增强的地标引导对比学习网络用于面部表情识别

LaCoVL-FER: Landmark-Guided Contrastive Learning Network with Vision-Language Enhancement for Facial Expression Recognition

School of Airspace and Engineering, Shandong University（山东大学航空航天与工程学院）；School of Computer Science and Technology, Shandong University of Finance and Economics（山东财经大学计算机科学与技术学院）；School of Psychology and Neuroscience, University of Glasgow（格拉斯哥大学心理学与神经科学学院）；Faculty of Information Science and Engineering, Ocean University of China（中国海洋大学信息科学与工程学院）

Jiaxin Wang, Muwei Jian, Hui Yu, Junyu Dong, Yifan Xia

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文提出了一种结合视觉-语言增强的地标引导对比学习网络LaCoVL-FER，通过引入面部地标几何先验和视觉-语言模型语义先验，解决野生环境中面部表情识别的挑战，提升识别的鲁棒性和泛化能力。

摘要：

在真实环境中，面部表情识别（FER）仍然具有挑战性，由于姿态、遮挡和光照的不可控变化。现有的基于注意力的方法主要依赖于视觉外观线索，导致注意力冗余和不稳定，限制了其在复杂场景中的性能。为了解决这些问题，我们提出了一种新颖的地标引导对比学习网络，结合视觉-语言增强，用于面部表情识别（LaCoVL-FER），该网络整合了来自面部地标几何先验和视觉-语言模型的语义先验。具体而言，设计了一个地标引导自适应编码器（LGAE），通过双分支门控交叉注意力（BGCA）机制引入几何先验，实现自适应融合基于地标几何和视觉外观特征，生成与表情相关的特征，从而聚焦于关键面部区域并抑制噪声干扰。同时，提出了一种视觉-语言增强策略（VLES），利用表情相关的特征来优化冻结预训练CLIP图像编码器提取的一般视觉特征，生成表情特定的视觉表示。基于这些表示，采用表情条件提示（ECP）机制进一步调整来自冻结预训练CLIP文本编码器的固定类级提示文本特征，生成更实例感知的文本表示。这些视觉-文本表示作为语义先验对齐，以增强FER的鲁棒性和泛化能力。定量和定性实验表明，我们的LaCoVL-FER在三个具有代表性的现实世界FER数据集（RAF-DB、FERPlus和AffectNet）上优于最先进的方法。代码可在https://github.com/ylin06804/LaCoVL-FER上获得。

英文摘要：

Facial Expression Recognition (FER) in the wild requires models to identify subtle expression cues under large variations in pose, occlusion, illumination, and identity. Recent FER methods improve robustness by introducing visual attention, facial landmarks, or vision-language models as auxiliary priors. However, these priors are typically integrated in a static manner, failing to capture instance-specific facial variations, thereby resulting in severe attention redundancy and representation instability. To address this issue, we propose LaCoVL-FER, a landmark-guided contrastive learning network with vision-language enhancement for FER, which shifts FER from static prior injection to sample-adaptive prior refinement. Specifically, a Landmark-Guided Adaptive Encoder (LGAE) calibrates regional appearance features with landmark geometry through Bi-branch Gated Cross Attention (BGCA), suppressing noisy responses and producing expression-relevant representations. In parallel, a Vision-Language Enhancement Strategy (VLES) refines the generalizable visual features from a frozen CLIP image encoder into expression-specific visual representations. Based on them, an Expression-Conditioned Prompting (ECP) mechanism adapts fixed class-level textual prompts from the frozen CLIP text encoder into instance-aware textual representations. The resulting visual-textual representations are aligned as adaptive semantic priors to enhance robustness and generalization. Quantitative and qualitative experiments show that LaCoVL-FER outperforms state-of-the-art methods on RAF-DB, FERPlus, and AffectNet. The code is available at https://github.com/ylin06804/LaCoVL-FER.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。