Although deep generative models have gained a lot of attention, most of the existing works are designed for unimodal generation. In this paper, we explore a new method for unconditional image-text pair generation. We design Multimodal Cross-Quantization VAE (MXQ-VAE), a novel vector quantizer for joint image-text representations, with which we discover that a joint image-text representation space is effective for semantically consistent image-text pair generation. To learn a multimodal semantic correlation in a quantized space, we combine VQ-VAE with a Transformer encoder and apply an input masking strategy. Specifically, MXQ-VAE accepts a masked image-text pair as input and learns a quantized joint representation space, so that the input can be converted to a unified code sequence, then we perform unconditional image-text pair generation with the code sequence. Extensive experiments show the correlation between the quantized joint space and the multimodal generation capability on synthetic and real-world datasets. In addition, we demonstrate the superiority of our approach in these two aspects over several baselines. The source code is publicly available at: https://github.com/ttumyche/MXQ-VAE.
Diffuse Everything: Multimodal Diffusion Models on Arbitrary State Spaces
Machine Learning Center, Georgia Institute of Technology, Atlanta, GA；School of Mathematics, Georgia Institute of Technology, Atlanta, GA；Department of Mathematics \& Statistics, SUNY Albany, NY
Straighter Flow Matching via a Diffusion-Based Coupling Prior
Institute of Information Engineering, Chinese Academy of Sciences（中国科学院信息工程研究所）；School of Cyber Security, University of Chinese Academy of Sciences（中国科学院大学网络空间安全学院）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）
Deep Generative Methods and Tire Architecture Design
ENS Paris-Saclay, Centre Borelli（巴黎-萨克雷大学ENS分校，Borelli中心）；Université Paris-Saclay, CNRS, ENS Paris-Saclay, Centre Borelli（巴黎-萨克雷大学，法国国家科学研究中心，巴黎-萨克雷大学ENS分校，Borelli中心）；ENSIIE（Évry法国的ENSIEE）
CMT: A Cascade MAR with Topology Predictor for Multimodal Conditional CAD Generation
Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）；Shanghai Jiao Tong University（上海交通大学）；The Chinese University of Hong Kong（香港中文大学）；Beihang University（北京航空航天大学）
SynCoGen: Synthesizable 3D Molecule Generation via Joint Reaction and Coordinate Modeling
University of Toronto（多伦多大学）；The Hospital for Sick Children（多伦多儿童医院）；University of Cambridge（剑桥大学）；ETH Zürich（苏黎世联邦理工学院）；Vector Institute（向量研究所）；Mila – Quebec AI Institute（魁北克AI研究所）；McGill University（麦吉尔大学）；Caltech（加州理工学院）
A Unified Framework for Multimodal Image Reconstruction and Synthesis using Denoising Diffusion Models
Department of Computer Science and Engineering, Washington University in St. Louis（华盛顿大学圣路易斯分校计算机科学与工程系）；Mallinckrodt Institute of Radiology, Washington University in St. Louis（华盛顿大学圣路易斯分校马林克罗德特放射医学研究所）；Department of Electrical and Systems Engineering, Washington University in St. Louis（华盛顿大学圣路易斯分校电气与系统工程系）；Department of Neurology, Washington University in St. Louis（华盛顿大学圣路易斯分校神经病学系）；Department of Biomedical Engineering, Washington University in St. Louis（华盛顿大学圣路易斯分校生物医学工程系）；Division of Biology and Biomedical Sciences, Washington University in St. Louis（华盛顿大学圣路易斯分校生物学与生物医学科学 division）；Department of Electrical and Computer Engineering, University of Wisconsin–Madison（威斯康星大学麦迪逊分校电气与计算机工程系）
TKCAM: Text and Keyframe to Camera Trajectory Generation
The University of Hong Kong（香港大学）；The Hong Kong University of Science and Technology（香港科技大学）；Macau University of Science and Technology（澳门科技大学）；Texas A&M University（德克萨斯农工大学）
