AULLM++：用于微表情动作单元检测的结构化令牌条件大语言模型
AULLM++: Structured-Token-Conditioned Large Language Models for Micro-Expression Action Unit Detection
AI总结：
针对微表情动作单元检测，以往方法有局限。本文提出AULLM++框架，利用大语言模型，将视觉特征注入文本提示，分证据构建、结构建模和基于推理预测三阶段，融合多粒度证据等，提升性能与泛化能力。
摘要：
微表情动作单元（AU）检测从微妙面部肌肉激活中识别局部AU，为解码情感线索奠定基础。先前方法存在三个关键限制：（1）严重依赖低密度视觉信息，使判别证据易受背景噪声影响；（2）粗粒度特征处理与细粒度表示需求不匹配；（3）忽视AU间相关性，限制复杂表情模式解析。我们提出AULLM++，这是一个利用大语言模型（LLMs）的面向推理的框架，它将视觉特征作为可操作的语义前提注入文本提示以指导推理。它将AU预测分为三个阶段：证据构建、结构建模和基于演绎的预测。具体来说，一个多粒度证据增强融合投影仪（MGE-EFP）将中级纹理线索与高级语义融合，将它们提炼成一个紧凑的内容令牌（CT）。此外，受微表情和宏表情AU对应关系的启发，我们将AU关系编码为稀疏结构先验，并通过关系感知AU图神经网络（R-AUGNN）学习交互强度，生成指令令牌（IT）。然后，我们将CT和IT融合成一个结构化文本提示，并引入反事实一致性正则化（CCR）来构建反事实样本，增强模型的泛化能力。大量实验表明，AULLM++在标准基准上取得了领先的性能，并表现出卓越的跨域泛化能力。
英文摘要：
Micro-expression Action Unit (AU) detection identifies localized AUs from subtle facial muscle activations, providing a foundation for decoding affective cues. Previous methods face three key limitations: (1) heavy reliance on low-density visual information, rendering discriminative evidence vulnerable to background noise; (2) coarse-grained feature processing that misaligns with the demand for fine-grained representations; and (3) neglect of inter-AU correlations, restricting the parsing of complex expression patterns. We propose AULLM++, a reasoning-oriented framework leveraging Large Language Models (LLMs), which injects visual features into textual prompts as actionable semantic premises to guide inference. It formulates AU prediction into three stages: evidence construction, structure modeling, and deduction-based prediction. Specifically, a Multi-Granularity Evidence-Enhanced Fusion Projector (MGE-EFP) fuses mid-level texture cues with high-level semantics, distilling them into a compact Content Token (CT). Furthermore, inspired by micro- and macro-expression AU correspondence, we encode AU relationships as a sparse structural prior and learn interaction strengths via a Relation-Aware AU Graph Neural Network (R-AUGNN), producing an Instruction Token (IT). We then fuse CT and IT into a structured textual prompt and introduce Counterfactual Consistency Regularization (CCR) to construct counterfactual samples, enhancing the model's generalization. Extensive experiments demonstrate AULLM++ achieves state-of-the-art performance on standard benchmarks and exhibits superior cross-domain generalization.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
