arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~

AffectFuse:用于多任务情感行为分析的具有时间建模的跨任务特征融合

AffectFuse: Cross-Task Feature Fusion with Temporal Modeling for Multi-Task Affective Behavior Analysis

Dipit Saha, Mohammad Raihan Rashid, Shah Mohammad Abdul Mannan, Ahnaf Tahmid, Md. Mehedi Hasan

arXiv 2607.16546首次发表:更新:

发表机构

Bangladesh University of Engineering and Technology(孟加拉国工程技术大学)

机构由 AI 辅助整理,请以论文原文为准。

AI 中文总结

针对自然环境情感行为识别,提出AffectFuse系统,通过编码器后适配,利用冻结主干提供特征,特定模块选择信号,采用MAE-Face与LoRA及专家路由识别动作单元,消融实验确定配置,有效构建MTL管道且无需训练新基础模型。

AI 中文摘要

在自然环境中的情感行为识别需要从无约束的面部图像中联合预测连续的效价-唤醒、分类面部表情和多标签动作单元。我们展示了针对第11届自然环境情感行为分析(ABAW)竞赛多任务学习(MTL)赛道中s-Aff-Wild2(Aff-Wild2的静态选定帧版本)的系统。该方法专注于编码器后适配:冻结的AffectNet监督主干提供多分辨率特征,而特定任务的时间头部和跨任务融合模块为每个目标选择有用信号。对于动作单元识别,我们采用带低秩适配(LoRA)的MAE-Face并通过每个单元的专家路由而非直接顺序转移来使用DISFA。通过对主干、时间、融合和AU适配选择进行消融实验来确定最终配置。最终系统在官方验证分割上获得P = 1.7302,表明编码器后适配和任务级建模选择提供了一个强大的MTL管道,而无需训练新的大规模面部基础模型。

英文摘要

Affective behavior recognition in the wild requires joint prediction of continuous valence-arousal, categorical facial expression, and multi-label action units from unconstrained face images. We present our system for the Multi-Task Learning (MTL) track of the 11th Affective Behavior Analysis in-the-wild (ABAW) competition on s-Aff-Wild2, the static selected-frame version of Aff-Wild2. The method focuses on post-encoder adaptation: frozen AffectNet-supervised backbones provide multi-resolution features, while task-specific temporal heads and cross-task fusion modules select the useful signals for each target. For action-unit recognition, we adapt MAE-Face with Low-Rank Adaptation (LoRA) and use DISFA through per-unit expert routing rather than direct sequential transfer. Ablations over backbone, temporal, fusion, and AU-adaptation choices define the final configuration. The final system obtains P = 1.7302 on the official validation split, showing that post-encoder adaptation and task-wise modeling choices provide a strong MTL pipeline without training a new large-scale face foundation model.

论文原文

arXiv 摘要页 · PDF 原文 · HTML 原文

↑