CAMO:基于因果引导的对抗多模态领域泛化用于危机分类
CAMO: Causality-Guided Adversarial Multimodal Domain Generalization for Crisis Classification
AI总结:
CAMO通过结合对抗去耦与统一表示学习,解决多模态危机分类中的领域泛化问题,提升模型在未见灾害场景下的性能。
摘要:
社交媒体中的危机分类旨在从多模态帖子中提取可操作的灾害相关信息,这是增强态势感知并促进及时应急响应的关键任务。然而,危机类型的广泛变化使得在未见的灾害上实现可推广的性能是一个持续的挑战。现有方法主要利用深度学习融合文本和视觉线索进行危机分类,在域内设置下实现了数值上合理的结果。然而,它们在未见的危机类型上表现不佳,因为它们1. 没有分离伪影和因果特征,导致在域转移下性能下降,2. 失败于在共享空间内对异构模态表示进行对齐,这阻碍了已建立的单模态领域泛化(DG)技术直接扩展到多模态设置。为了解决这些问题,我们引入了一个基于因果引导的多模态领域泛化(MMDG)框架,该框架结合对抗去耦与统一表示学习进行危机分类。对抗目标鼓励模型分离并专注于域不变的因果特征,从而得到基于稳定因果机制的更具推广性的分类。统一的表示将不同模态的特征在共享的潜在空间中对齐,使单模态DG策略能够无缝扩展到多模态学习。在不同数据集上的实验表明,我们的方法在未见的灾害场景中实现了最佳性能。
英文摘要:
Crisis classification in social media aims to extract actionable disaster-related information from multimodal posts, which is a crucial task for enhancing situational awareness and facilitating timely emergency responses. However, the wide variation in crisis types makes achieving generalizable performance across unseen disasters a persistent challenge. Existing approaches primarily leverage deep learning to fuse textual and visual cues for crisis classification, achieving numerically plausible results under in-domain settings. However, they exhibit poor generalization across unseen crisis types because they 1. do not disentangle spurious and causal features, resulting in performance degradation under domain shift, and 2. fail to align heterogeneous modality representations within a shared space, which hinders the direct adaptation of established single-modality domain generalization (DG) techniques to the multimodal setting. To address these issues, we introduce a causality-guided multimodal domain generalization (MMDG) framework that combines adversarial disentanglement with unified representation learning for crisis classification. The adversarial objective encourages the model to disentangle and focus on domain-invariant causal features, leading to more generalizable classifications grounded in stable causal mechanisms. The unified representation aligns features from different modalities within a shared latent space, enabling single-modality DG strategies to be seamlessly extended to multimodal learning. Experiments on the different datasets demonstrate that our approach achieves the best performance in unseen disaster scenarios.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。