Diffusion Models for Graphs Benefit From Discrete State Spaces
ETH Zurich(苏黎世联邦理工学院);Swiss Data Science Center(瑞士数据科学中心)
Kilian Konstantin Haefeli, Karolis Martinkus, Nathanaël Perraudin, Roger Wattenhofer
备注:
Presented at the First Learning on Graphs Conference (LoG 2022) and the NeurIPS 2022 New Frontiers in Graph Learning Workshop (NeurIPS GLFrontiers 2022)
Denoising diffusion probabilistic models and score-matching models have proven to be very powerful for generative tasks. While these approaches have also been applied to the generation of discrete graphs, they have, so far, relied on continuous Gaussian perturbations. Instead, in this work, we suggest using discrete noise for the forward Markov process. This ensures that in every intermediate step the graph remains discrete. Compared to the previous approach, our experimental results on four datasets and multiple architectures show that using a discrete noising process results in higher quality generated samples indicated with an average MMDs reduced by a factor of 1.5. Furthermore, the number of denoising steps is reduced from 1000 to 32 steps, leading to a 30 times faster sampling procedure.
Inria(法国国家信息与自动化技术研究院);PSL Research University(巴黎社会科学高等研究院);CMAP(巴黎高等理工学院应用数学与计算科学实验室);Ecole Polytechnique Palaiseau, France(法国巴黎高等理工学院Palaiseau分校);Sakana AI Tokyo, Japan(日本东京Sakana AI公司)
Variational Autoencoding Discrete Diffusion with Enhanced Dimensional Correlations Modeling
School of Mathematical Sciences, Peking University(北京大学数学科学学院);Center for Statistical Science, Peking University(北京大学统计科学中心);Academy of Mathematics and Systems Science, Chinese Academy of Sciences(中国科学院数学与系统科学研究院);Huawei Foundation Model Dept(华为基金会模型部门);The Chinese University of Hong Kong, Shenzhen(香港中文大学(深圳))
Discrete Diffusion Models via Evolving Variational Autoregressive Networks
Institute of Fundamental and Frontier Sciences, University of Electronic Science and Technology of China(电子科技大学基础与前沿研究院);School of Physics, University of Electronic Science and Technology of China(电子科技大学物理学院);Key Laboratory of Quantum Physics and Photonic Quantum Information, Ministry of Education, University of Electronic Science and Technology of China(电子科技大学教育部量子物理与光子量子信息重点实验室);Non-classical Information Science Basic Discipline Research Center of Sichuan Province, University of Electronic Science and Technology of China(电子科技大学四川省非经典信息科学基础学科研究中心)
Interpolating Discrete Diffusion Models with Controllable Resampling
School of Computation, Information and Technology, Technical University of Munich(计算、信息与技术学院,慕尼黑技术大学);Munich Data Science Institute(慕尼黑数据科学研究所);Munich Center for Machine Learning(慕尼黑机器学习中心)
Unifying Continuous and Discrete Text Diffusion with Non-simultaneous Diffusion Processes
School of Computer Science and Technology, University of Science and Technology of China(计算机科学与技术学院,中国科学技术大学);State Key Laboratory of Cognitive Intelligence(认知智能国家重点实验室)
Simple and Critical Iterative Denoising: A Recasting of Discrete Diffusion in Graph Generation
Data Mining and Machine Learning group, School for Business Administration (HEG-Geneva), University of Applied Sciences and Arts of Western Switzerland(应用科学与艺术大学西瑞士分校数据挖掘与机器学习小组);Computer Science Department, University of Geneva, Switzerland(日内瓦大学计算机科学系)
A2D2: Fine-Tuning Any-Length Discrete Diffusion for Adaptive Decoding
Department of Computer and Information Science, University of Pennsylvania(宾夕法尼亚大学计算机与信息科学系);Department of Bioengineering, University of Pennsylvania(宾夕法尼亚大学生物工程系);School of Mathematics, Georgia Institute of Technology(佐治亚理工学院数学学院)
DICE: Discrete Inversion Enabling Controllable Editing for Multinomial Diffusion and Masked Generative Models
Rutgers University(新泽西罗格斯大学);MIT-IBM Watson AI Lab(MIT-IBM沃森人工智能实验室);Red Hat AI Innovation(红帽AI创新);Google DeepMind(谷歌DeepMind);NYU(纽约大学);Walmart Global Tech(沃尔玛全球技术);NEC Labs America(NEC美国实验室);Massachusetts Institute of Technology(麻省理工学院);ANU(澳大利亚国立大学);UT Arlington(德克萨斯大学阿灵顿分校)