Vision-Language Models (VLMs) are typically trained on a diverse set of multi-modal domains, yet current practices rely on costly manual tuning. We propose MaD-Mix, a principled and computationally efficient framework that derives multi-modal data mixtures for VLM training. MaD-Mix formulates data mixing as modality-aware domain alignment maximization and obtains closed-form multi-modal alignment scores from the Fenchel dual through inter-modal coupling variables. MaD-Mix systematically handles domains with missing modalities, allowing for the integration of language-only domains. Empirical evaluations across 0.5B and 7B models demonstrate that MaD-Mix accelerates VLM training across diverse benchmarks. MaD-Mix matches human-tuned data mixtures using 22% fewer training steps in image-text instruction tuning. In complex tri-modal video-image-text scenarios, where manual tuning becomes impractical, MaD-Mix boosts average accuracy over uniform weights, with negligible mixture computation overhead (< 1 GPU-hour), enabling scalable mixture design for modern VLM pipelines.
Improving Normative Modeling for Multi-modal Neuroimaging Data using mixture-of-product-of-experts variational autoencoders
Department of Computer Science and Engineering, Washington University in St. Louis, USA（计算机科学与工程系，华盛顿大学圣路易斯分校）；Institute for Informatics, Data Science and Biostatistics, Washington University in St.Louis, USA（信息学、数据科学与生物统计研究所，华盛顿大学圣路易斯分校）；Department of Radiology, Washington University in St.Louis, USA（放射学系，华盛顿大学圣路易斯分校）
Fraunhofer SIT at CheckThat! 2023: Mixing Single-Modal Classifiers to Estimate the Check-Worthiness of Multi-Modal Tweets
Fraunhofer Institute for Secure Information Technology SIT（弗劳恩霍夫安全信息技术研究所）；ATHENE - National Research Center for Applied Cybersecurity（ATHENE——国家应用网络安全研究中心）
Unified Cross-Modal Medical Image Synthesis with Hierarchical Mixture of Product-of-Experts
Harvard Medical School and the Brigham and Women’s Hospital（哈佛医学院和布里洛妇产科医院）；MIND Team, Inria Saclay, Université Paris-Saclay, Palaiseau, France and Sorbonne Université, Institut du Cerveau - Paris Brain Institute - ICM, CNRS, Inria, Inserm, AP-HP, Hôpital de la Pitié Salpêtrière（MIND团队、Inria Saclay、巴黎萨克雷大学、Palaiseau，法国以及索邦大学、巴黎脑研究所-ICM、CNRS、Inria、Inserm、AP-HP、皮蒂埃萨勒普蒂耶医院）；Massachusetts Institute of Technology（麻省理工学院）
NeuroMoE: A Transformer-Based Mixture-of-Experts Framework for Multi-Modal Neurological Disorder Classification
Department of Electrical and Computer Engineering, University of Houston（电气与计算机工程系，休斯顿大学）；Department of Information Science Technology, University of Houston（信息科学与技术系，休斯顿大学）；Movement Disorders & Neurodegenerative Diseases Division, University of Texas Health Science Center at Houston（运动障碍与神经退行性疾病部门，德克萨斯大学健康科学中心休斯顿分部）；Department of Neurology at McGovern Medical School, University of Texas Health Science Center at Houston（神经病学系，麦戈文医学学校，德克萨斯大学健康科学中心休斯顿分部）；Department of Psychology, The City College of New York（心理学系，纽约城市大学）
MixEval-X: Any-to-Any Evaluations from Real-World Data Mixtures
National University of Singapore（新加坡国立大学）；Peking University（北京大学）；Google DeepMind（谷歌DeepMind）；Nanyang Technological University（南洋理工大学）；Carnegie Mellon University（卡内基梅隆大学）；University of Waterloo（滑铁卢大学）
From Consistency to Complementarity: Aligned and Disentangled Multi-modal Learning for Time Series Understanding and Reasoning
The Hong Kong University of Science（香港科学与技术大学）；Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）；University of Chinese Academy of Sciences（中国科学院大学）
