面向隐私安全的非个体化方法的多模态群体情绪识别
Multimodal Group Emotion Recognition In-the-Wild Towards a Privacy-Safe Non-Individual Approach
AI总结：
本文提出两种多模态框架（交叉注意力融合+帧注意力池化，以及变分编码器多解码器），利用集体音视频信号进行群体情绪识别，避免使用个体特征，在保护隐私的同时实现鲁棒性能。
摘要：
本论文研究野外环境下的群体情绪识别（GER），重点关注隐私保护。与依赖面部、目光或语音分析等个体层面线索的传统情绪识别方法不同，本工作利用集体音视频信号推断群体层面的情绪，降低个体监控和监视的风险。提出了两个互补框架。第一个是用于音视频融合的交叉注意力多模态架构，结合帧注意力池化（FAP）进行时间聚合。该框架由合成数据增强支持，并通过消融研究验证，在真实世界GER条件下展现出鲁棒性。第二个框架，变分编码器多解码器（VE-MD），学习一个共享潜在空间，用于情绪分类和结构表示预测（包括身体和面部线索）。探索了两种解码策略（基于DETR和基于热图），以分析结构表示在群体和个体设置中的作用。本论文做出三项主要贡献：阐明了多模态和结构线索在群体层面情感计算中的作用；引入了两种用于隐私保护多模态GER的架构；并证明了在不使用个体特征作为输入数据的情况下可以实现有竞争力的性能。
英文摘要：
This thesis addresses group emotion recognition (GER) in-the-wild with a focus on privacy preservation. Unlike traditional emotion recognition methods that rely on individual-level cues such as face, gaze, or voice analysis, this work uses collective audio-video signals to infer emotions at the group level, reducing risks of individual monitoring and surveillance. Two complementary frameworks are proposed. The first is a cross-attention multimodal architecture for audio-video fusion, combined with Frames Attention Pooling (FAP) for temporal aggregation. It is supported by synthetic data augmentation and validated through ablation studies, demonstrating robustness in real-world GER conditions. The second framework, Variational Encoder Multi-Decoder (VE-MD), learns a shared latent space for emotion classification and structural representation prediction, including body and face cues. Two decoding strategies, DETR-based and heatmap-based, are explored to analyze the role of structural representations in group and individual settings. The thesis makes three main contributions: it clarifies the role of multimodality and structural cues in group-level affective computing; introduces two architectures for privacy-preserving multimodal GER; and shows that competitive performance can be achieved without using individual features as input data.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
