DeepMind(深度思维);Google Research(谷歌研究院);University of Southern California(南加州大学);INRIA(法国国家信息与自动化研究所);Ecole Normale Supérieure(巴黎高等师范学院)
Sander Dieleman, Laurent Sartran, Arman Roshannai, Nikolay Savinov, Yaroslav Ganin, Pierre H. Richemond, Arnaud Doucet, Robin Strudel, Chris Dyer, Conor Durkan,… 展开作者
Sander Dieleman, Laurent Sartran, Arman Roshannai, Nikolay Savinov, Yaroslav Ganin, Pierre H. Richemond, Arnaud Doucet, Robin Strudel, Chris Dyer, Conor Durkan, Curtis Hawthorne, Rémi Leblond, Will Grathwohl, Jonas Adler
备注:
26 pages, 8 figures; corrections and additional information about hyperparameters
Diffusion models have quickly become the go-to paradigm for generative modelling of perceptual signals (such as images and sound) through iterative refinement. Their success hinges on the fact that the underlying physical phenomena are continuous. For inherently discrete and categorical data such as language, various diffusion-inspired alternatives have been proposed. However, the continuous nature of diffusion models conveys many benefits, and in this work we endeavour to preserve it. We propose CDCD, a framework for modelling categorical data with diffusion models that are continuous both in time and input space. We demonstrate its efficacy on several language modelling tasks.
Optimality of FSQ Tokens for Continuous Diffusion for Categorical Data with Application to Text-to-Speech
Huawei Noah's Ark Lab(华为诺亚实验室);National Research University Higher School of Economics(俄罗斯国家研究大学高等经济学院);National University of Science and Technology MISIS(俄罗斯科学与技术国立大学MISIS)
UvA-Bosch Delta Lab, University of Amsterdam, Amsterdam, Netherlands(阿姆斯特丹大学);Department of Computer Science, University of Oxford, Oxford, UK(牛津大学计算机科学系);Donders Institute for Brain, Cognition and Behaviour, Radboud University(拉德堡德大学大脑与行为研究所)
Simplex-to-Euclidean Bijections for Categorical Flow Matching
Department of Computer Science, University of Helsinki(赫尔辛基大学计算机科学系);Department of Computer Science, Aalto University(阿莱大学计算机科学系);ELLIS Institute Finland(芬兰ELLIS研究所)
Foundations of Diffusion Models in General State Spaces: A Self-Contained Introduction
Technical University of Munich(慕尼黑工业大学);Helmholtz AI(亥姆霍兹AI);Munich Center for Machine Learning (MCML)(慕尼黑机器学习中心);Mila – Quebec AI Institute(Mila-魁北克人工智能研究所);AITHYRA
CDMT-EHR: A Continuous-Time Diffusion Framework for Generating Mixed-Type Time-Series Electronic Health Records
Department of Core Informatics, Graduate School of Informatics, Osaka Metropolitan University(核心信息学系,信息学研究生学校,大阪 Metropolitan 大学);Smart Data & Knowledge Services Department, German Research Center for Artificial Intelligence (DFKI GmbH)(智能数据与知识服务部,德国人工智能研究中心(DFKI GmbH));Department of Hematology and Department of Laboratory Medicine and Medical Informatics, Graduate School of Medicine, Osaka Metropolitan University(血液科和实验室医学与医学信息学系,医学研究生学校,大阪 Metropolitan 大学)
NTU Singapore(新加坡南洋理工大学);The University of Tokyo(东京大学);Purdue University(普渡大学);Institute for Advanced Intelligence and Computing (IAIC), A*STAR(新加坡科技研究局高级智能与计算研究所);Centre for Frontier AI Research (CFAR), A*STAR(新加坡科技研究局前沿人工智能研究中心)