Latent diffusion models have emerged as the dominant framework for high-fidelity and efficient image generation, owing to their ability to learn diffusion processes in compact latent spaces. However, while previous research has focused primarily on reconstruction accuracy and semantic alignment of the latent space, we observe that another critical factor, robustness to sampling perturbations, also plays a crucial role in determining generation quality. Through empirical and theoretical analyses, we show that the commonly used $β$-VAE-based tokenizers in latent diffusion models, tend to produce overly compact latent manifolds that are highly sensitive to stochastic perturbations during diffusion sampling, leading to visual degradation. To address this issue, we propose a simple yet effective solution that constructs a latent space robust to sampling perturbations while maintaining strong reconstruction fidelity. This is achieved by introducing a Variance Expansion loss that counteracts variance collapse and leverages the adversarial interplay between reconstruction and variance expansion to achieve an adaptive balance that preserves reconstruction accuracy while improving robustness to stochastic sampling. Extensive experiments demonstrate that our approach consistently enhances generation quality across different latent diffusion architectures, confirming that robustness in latent space is a key missing ingredient for stable and faithful diffusion sampling.
VideoRAE: Taming Video Foundation Models for Generative Modeling via Representation Autoencoders
The Chinese University of Hong Kong, Shenzhen(香港中文大学(深圳));Huazhong University of Science and Technology(华中科技大学);Shenzhen Loop Area Institute(深圳河套学院);University of Science and Technology of China(中国科学技术大学)
Juno: Taming Predictive Latents for Vision-Language-Action Models
University of Science and Technology of China(中国科学技术大学);Eastern Institute of Technology, Ningbo(宁波东方理工大学);Hangzhou Dianzi University(杭州电子科技大学);ShanghaiTech University(上海科技大学)
TReFT: Taming Rectified Flow Models For One-Step Image Translation
University of Chinese Academy of Sciences(中国科学院大学);Institute of Computing Technology, Chinese Academy of Sciences(中国科学院计算技术研究所);Beihang University(北航);Tianjin University(天津大学);CreateAI
Edit-VAR: Taming Visual Autoregressive Model for Precise Video Editing
Sun Yat-sen University(中山大学);South China University of Technology(华南理工大学);Tsinghua University(清华大学);Shandong University(山东大学);Nankai University(南开大学);Hunan University(湖南大学)
Taming Generative Synthetic Data for X-ray Prohibited Item Detection
Institute of Information Science, Beijing Jiaotong University(信息科学学院,北京交通大学);Faculty of Data Science, City University of Macau(数据科学学院,澳门城市大学);Nuctech Company Limited(NuTech公司)
VoRTeC: Taming Foundation Flow for One-step Real time Video Compression
Shenzhen International Graduate School(深圳国际研究生院);Tsinghua University(清华大学);Harbin Institute of Technology, Shenzhen(哈尔滨工业大学(深圳));Peking University(北京大学)
Taming the Tri-Space Tension: ARC-Guided Hallucination Modeling and Control for Text-to-Image Generation
University of Bristol(布里斯托大学);South China University of Technology(华南理工大学);Brown University(布朗大学);Xi’an Jiaotong University(西安交通大学);Arizona State University(亚利桑那州立大学)
OneWorld: Taming Scene Generation with 3D Unified Representation Autoencoder
Mohamed bin Zayed University of Artificial Intelligence(莫扎德·本·泽亚德人工智能大学);AISphere;Shanghai Jiao Tong University(上海交通大学);University of Melbourne(墨尔本大学);Nanyang Technological University(南洋理工大学)
DiffVSR: Revealing an Effective Recipe for Taming Robust Video Super-Resolution Against Complex Degradations
Shanghai Jiao Tong University(上海交通大学);Shanghai Artificial Intelligence Laboratory(上海人工智能实验室);University of Science and Technology of China(中国科学技术大学);Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences(中国科学院深圳先进技术研究院)
Taming Real-World Space-Time Video Super-Resolution with One-Step Diffusion
Institute of Information Science, Beijing Jiaotong University(北京交通大学信息科学学院);Visual Intelligence + X International Cooperation Joint Laboratory of MOE, Beijing(教育部视觉智能+X国际合作联合实验室);Innovation School of Artificial Intelligence, Hefei University of Technology(合肥工业大学人工智能创新学院);School of Control Science and Engineering, Shandong University(山东大学控制科学与工程学院)