In real-world scenarios, achieving domain generalization (DG) presents significant challenges as models are required to generalize to unknown target distributions. Generalizing to unseen multi-modal distributions poses even greater difficulties due to the distinct properties exhibited by different modalities. To overcome the challenges of achieving domain generalization in multi-modal scenarios, we propose SimMMDG, a simple yet effective multi-modal DG framework. We argue that mapping features from different modalities into the same embedding space impedes model generalization. To address this, we propose splitting the features within each modality into modality-specific and modality-shared components. We employ supervised contrastive learning on the modality-shared features to ensure they possess joint properties and impose distance constraints on modality-specific features to promote diversity. In addition, we introduce a cross-modal translation module to regularize the learned features, which can also be used for missing-modality generalization. We demonstrate that our framework is theoretically well-supported and achieves strong performance in multi-modal DG on the EPIC-Kitchens dataset and the novel Human-Animal-Cartoon (HAC) dataset introduced in this paper. Our source code and HAC dataset are available at https://github.com/donghao51/SimMMDG.
Adaptive Physical-Facial Representation Fusion via Subject-Invariant Cross-Modal Prompt Tuning for Video-Based Emotion Recognition
Department of Biomedical Engineering（生物医学工程系）；Anhui Province Key Laboratory of Measuring Theory and Precision Instrument（安徽省测量理论与精密仪器重点实验室）；School of Computer Science and Information Engineering（计算机科学与信息工程学院）
ViLPAct: A Benchmark for Compositional Generalization on Multimodal Human Activities
Monash University（莫纳什大学）；University of Electronic Science and Technology of China（电子科技大学）；University of Technology Sydney（悉尼科技大学）；Harbin Institute of Technology, Shenzhen（哈尔滨工业大学（深圳））；Peng Cheng Lab（鹏城实验室）
Any2Any: Unified Arbitrary Modality Translation for Remote Sensing
National Engineering Research Center for Multimedia Software, Institute of Artificial Intelligence, School of Computer Science, Wuhan University（多媒体软件国家工程研究中心、人工智能研究院、计算机科学学院、武汉大学）；Hubei Key Laboratory of Multimedia（湖北省多媒体重点实验室）；Zhongguancun Academy, Beijing, China. 100094（中关村学院，北京，中国。100094）；School of Electronic Information, Wuhan University, Wuhan, China（电子信息学院，武汉大学，武汉，中国）；School of Automation, Beijing Institute of Technology（自动化学院，北京理工大学）；State Key Laboratory of Information Engineering in Surveying, Mapping and Remote Sensing, Wuhan University, Wuhan, China（测绘、制图与遥感信息工程国家重点实验室，武汉大学，武汉，中国）
REVELIO: Universal Multimodal Task Load Estimation for Cross-Domain Generalization
Fraunhofer Institute for Integrated Circuits IIS（弗劳恩霍夫集成电路研究所）；Friedrich-Alexander-University Erlangen Nuremberg（埃朗根-纽伦堡弗赖堡-亚历山大大学）；Helmholtz Zentrum München（亥姆霍兹慕尼黑研究中心）
Towards a Universal 3D Medical Multi-modality Generalization via Learning Personalized Invariant Representation
Shanghai Academy of Artificial Intelligence for Science（上海人工智能科学研究院）；Xi’an Jiaotong-Liverpool University（西安交通大学利物浦大学）；University of Liverpool（利物浦大学）；AI 3 Fudan University（复旦大学）；Duke Kunshan University（杜克-昆山大学）；Zhongshan Hospital（中山医院）
Cross-Modal-Domain Generalization Through Semantically Aligned Discrete Representations
Peter L. Reichertz Institute for Medical Informatics（汉诺威医学院彼得·L·里赫茨医学信息学研究所）；Lower Saxony Center for AI and Causal Methods in Medicine (CAIMed)（下萨克森医学人工智能与因果方法中心（CAIMed））
Variational Adapter for Cross-modal Similarity Representation
School of Remote Sensing and Information Engineering（遥感与信息工程学院）；Wuhan University（武汉大学）；School of Data Science and Engineering（数据科学与工程学院）；East China Normal University（华东师范大学）；State Key Laboratory of Information Engineering in Surveying, Mapping and Remote Sensing（测绘遥感信息工程国家重点实验室）
Robust Domain Generalization for Multi-modal Object Recognition
Northern Arizona University（北亚利桑那大学）；AMA University（AMA大学）；Massachusetts Institute of Technology（麻省理工学院）；Peking University（北京大学）；The University of Texas at Austin（德克萨斯大学奥斯汀分校）；University of Southern California（南加州大学）；College of Computing Georgia Institute of Technology（佐治亚理工学院计算学院）
CIBR: Cross-modal Information Bottleneck Regularization for Robust CLIP Generalization
Aerospace Information Research Institute, Chinese Academy of Sciences（中国科学院空天信息创新研究院）；University of Alabama at Birmingham（阿拉巴马大学伯明翰分校）；Harvard Medical School（哈佛医学院）；Wuhan University（武汉大学）；Bowling Green State University（鲍灵格林州立大学）；University of California, Irvine（加州大学欧文分校）；School of Electronic, Electrical and Communication Engineering, University of Chinese Academy of Sciences（中国科学院大学电子电气与通信工程学院）
