A Phase Transition in Diffusion Models Reveals the Hierarchical Nature of Data
Institute of Physics, Ecole Polytechnique Fédérale de Lausanne(洛桑联邦理工学院物理研究所);Institute of Electrical and Micro Engineering, Ecole Polytechnique Fédérale de Lausanne(洛桑联邦理工学院电气与微工程研究所)
Antonio Sclocchi, Alessandro Favero, Matthieu Wyart
Understanding the structure of real data is paramount in advancing modern deep-learning methodologies. Natural data such as images are believed to be composed of features organized in a hierarchical and combinatorial manner, which neural networks capture during learning. Recent advancements show that diffusion models can generate high-quality images, hinting at their ability to capture this underlying compositional structure. We study this phenomenon in a hierarchical generative model of data. We find that the backward diffusion process acting after a time $t$ is governed by a phase transition at some threshold time, where the probability of reconstructing high-level features, like the class of an image, suddenly drops. Instead, the reconstruction of low-level features, such as specific details of an image, evolves smoothly across the whole diffusion process. This result implies that at times beyond the transition, the class has changed, but the generated sample may still be composed of low-level elements of the initial image. We validate these theoretical insights through numerical experiments on class-unconditional ImageNet diffusion models. Our analysis characterizes the relationship between time and scale in diffusion models and puts forward generative models as powerful tools to model combinatorial data properties.
Generate the Forest before the Trees - A Hierarchical Diffusion model for Climate Downscaling
School of Computer Science and Engineering, University of New South Wales(计算机科学与工程学院,新南威尔士大学);ARC Centre of Excellence for the Weather of the 21 st Century and Climate Change Research Centre, University of New South Wales(21世纪天气卓越研究中心及气候变化研究中心,新南威尔士大学)
Swinv2-Imagen: Hierarchical Vision Transformer Diffusion Models for Text-to-Image Generation
Auckland University of Technology(奥克兰理工大学);Hefei University of Technology(合肥工业大学);University of Tasmania(塔斯马尼亚大学);Jilin University of Finance and Economics(吉林财经大学)
School of Electronic and Computer Engineering, Peking University(北京大学电子与计算机工程学院);Peng Cheng Laboratory(鹏城实验室);Peking University Shenzhen Graduate School-Rabbitpre AIGC Joint Research Laboratory(北京大学深圳研究生院-睿魔智能AIGC联合实验室)
Hierarchical Conditioning of Diffusion Models Using Tree-of-Life for Studying Species Evolution
Virginia Tech(弗吉尼亚理工大学);Oak Ridge National Laboratory(橡树岭国家实验室);University of California, Irvine(加利福尼亚大学欧文分校);Tulane University(杜兰大学);Battelle(巴特尔公司);Duke University(杜克大学);University of North Carolina at Chapel Hill(北卡罗来纳大学教堂山分校);The Ohio State University(俄亥俄州立大学);Rensselaer Polytechnic Institute(伦斯勒理工学院)
HAVIR: HierArchical Vision to Image Reconstruction using CLIP-Guided Versatile Diffusion
Research Center for Medical AI, Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences(医学人工智能研究中心,深圳先进技术研究所,中国科学院);Research Center for Biomedical Imaging, Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences(生物医学成像研究中心,深圳先进技术研究所,中国科学院)
Hierarchical Semi-Implicit Variational Inference with Application to Diffusion Model Acceleration
Peking University(北京大学);Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);Beijing Academy of Artificial Intelligence(北京人工智能研究院);Megvii Technology Inc.(旷视科技有限公司)
Hierarchical Integration Diffusion Model for Realistic Image Deblurring
Shanghai Jiao Tong University(上海交通大学);ETH Zürich(苏黎世联邦理工学院);Bytedance Inc(字节跳动公司);Tsinghua University(清华大学);Shanghai AI Laboratory(上海人工智能实验室);The University of Sydney(悉尼大学);Westlake University(西湖大学)
Hierarchical Diffusion Framework for Pseudo-Healthy Brain MRI Inpainting with Enhanced 3D Consistency
The Grainger College of Engineering(格拉inger工程学院);National Center for Supercomputing Applications(国家超级计算应用中心);University of Illinois Urbana‑Champaign(伊利诺伊大学厄巴纳-香槟分校)
H-LDM: Hierarchical Latent Diffusion Models for Controllable and Interpretable PCG Synthesis from Clinical Metadata
School of Cyber Engineering Xidian University Xi'an, China(电子科技大学信息工程学院西安中国);School of Telecommunications Engineering Xidian University Xi'an, China(电子科技大学电信工程学院西安中国)
HieraFashDiff: Hierarchical Fashion Design with Multi-stage Diffusion Models
Shanghai University(上海大学);ShanghaiTech University(上海科技大学);Giant Network(巨人网络);Shanghai Engineering Research Center of Motion Picture Special Effects(上海电影特效工程技术研究中心)
NeuralField-LDM: Scene Generation with Hierarchical Latent Diffusion Models
NVIDIA(英伟达);University of Toronto(多伦多大学);Vector Institute(向量研究所);CSAIL, MIT(麻省理工学院计算机科学与人工智能实验室);University of Waterloo(滑铁卢大学);University of Tübingen(图宾根大学);Tübingen AI Center(图宾根人工智能中心);LMU Munich(慕尼黑大学)
Hierarchical Denoising For Multi-Step Visual Reasoning
State Key Laboratory of Multimedia Information Processing, School of Computer Science, Peking University(北京大学计算机科学学院多媒体信息处理技术国家重点实验室);The Hong Kong University of Science and Technology(香港科技大学);Beihang University(北京航空航天大学);Fuzhou University(福州大学);Muka Robotics(木卡机器人)
SHMT: Self-supervised Hierarchical Makeup Transfer via Latent Diffusion Models
School of Computer Science and Artificial Intelligence, Wuhan University of Technology(武汉理工大学计算机科学与人工智能学院);Sanya Science and Education Innovation Park, Wuhan University of Technology(武汉理工大学三亚科教创新园);DAMO Academy, Alibaba Group(阿里巴巴达摩院);Shanghai AI Laboratory(上海人工智能实验室)
H2-Cache: A Novel Hierarchical Dual-Stage Cache for High-Performance Acceleration of Generative Diffusion Models
Department of Artificial Intelligence, Kyungpook National University(人工智能系,庆尚国立大学);Department of Electrical and Computer Engineering, Queen’s University(电气与计算机工程系,皇后大学);Department of Information and Communications Engineering, Pukyong National University(信息与通信工程系,浦项国立大学)
Hierarchical Schedule Optimization for Fast and Robust Diffusion Model Sampling
School of Computer Science and Engineering, Macau University of Science and Technology(澳门科学技术大学计算机科学与工程学院);Beijing Institute of Technology(北京理工大学);Zhejiang University(浙江大学)