发表机构
Shandong University; University of Glasgow; Institute of Computing Technology, Chinese Academy of Sciences（山东大学; 格拉斯哥大学; 中国科学院计算技术研究所）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
本综述系统梳理多模态面部状态分析的任务、方法与数据集，强调多模态学习与多任务学习在提升推理、可解释性及跨场景泛化中的关键作用，并展望未来方向。
AI 中文摘要
面部状态分析在理解人类表情、心理建模和人机交互中起着至关重要的作用。传统的基于视觉的单模态方法往往受限于环境敏感性和较弱的可解释性。多模态面部状态分析通过整合来自视觉、音频、文本、生理及其他相关模态的互补线索来解决这些问题。本综述强调两个关键方面：一方面，多模态学习能够实现上下文语义理解，以改进面部状态推理，并利用可解释的语言生成来增强模型的可解释性；另一方面，多任务学习允许同时分析表情、动作单元（AUs）以及基于面部的软生物特征（如年龄、性别），有效捕捉细粒度表情并提高跨场景泛化能力。本综述回顾了多模态面部状态分析中的核心任务、代表性方法和数据集，重点关注面部表情识别、AU检测和基于面部的软生物特征估计，并强调语言在提供上下文语义、增强推理和生成解释方面的独特价值。本综述旨在提供文献的最新概述，并突出多模态、可解释和自适应多任务面部状态分析的未来研究方向。
英文摘要
Facial state analysis plays a crucial role in understanding human expressions, psychological modeling, and human computer interaction. Traditional unimodal vision-based methods are often limited by environmental sensitivity and weak interpretability. Multimodal facial state analysis addresses these issues by integrating complementary cues from visual, audio, textual, physiological, and other related modalities. This survey emphasizes two key aspects: on one hand, multimodal learning enables contextual semantic understanding for improved facial state reasoning and leverages interpretable language generation to enhance model explainability; on the other hand, multi-task learning allows simultaneous analysis of expressions, action units (AUs), and face-based soft biometrics (e.g., age, gender), effectively capturing fine-grained expressions and improving cross-scene generalization. This survey reviews core tasks, representative methods, and datasets in multimodal facial state analysis, focusing on facial expression recognition, AU detection, and face-based soft biometric estimation, and emphasizing the unique value of language in providing contextual semantics, enhancing reasoning, and generating explanations. The survey aims to provide an up-to-date overview of the literature and to highlight future research directions for multimodal, interpretable, and multi-task adaptive facial state analysis.
Journal refICME2026
