arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2211.15089cs.CLcs.LG
收藏

分类数据的连续扩散

Continuous diffusion for categorical data

DeepMind(深度思维);Google Research(谷歌研究院);University of Southern California(南加州大学);INRIA(法国国家信息与自动化研究所);Ecole Normale Supérieure(巴黎高等师范学院)
Sander Dieleman, Laurent Sartran, Arman Roshannai, Nikolay Savinov, Yaroslav Ganin, Pierre H. Richemond, Arnaud Doucet, Robin Strudel, Chris Dyer, Conor Durkan,… 展开作者

Sander Dieleman, Laurent Sartran, Arman Roshannai, Nikolay Savinov, Yaroslav Ganin, Pierre H. Richemond, Arnaud Doucet, Robin Strudel, Chris Dyer, Conor Durkan, Curtis Hawthorne, Rémi Leblond, Will Grathwohl, Jonas Adler

摘要:

扩散模型通过迭代细化已迅速成为感知信号(如图像和声音)生成建模的首选范例。他们的成功取决于这样一个事实,即潜在的物理现象是连续的。对于固有的离散和分类数据,如语言,已经提出了各种扩散启发的替代方案。然而,连续性质的扩散模型传达了许多好处,在这项工作中,我们努力保护it.We提出CDCD,一个框架,用于建模的分类数据与扩散模型是连续的时间和输入空间。我们在几个语言建模任务上证明了它的有效性。

英文摘要:

Diffusion models have quickly become the go-to paradigm for generative modelling of perceptual signals (such as images and sound) through iterative refinement. Their success hinges on the fact that the underlying physical phenomena are continuous. For inherently discrete and categorical data such as language, various diffusion-inspired alternatives have been proposed. However, the continuous nature of diffusion models conveys many benefits, and in this work we endeavour to preserve it. We propose CDCD, a framework for modelling categorical data with diffusion models that are continuous both in time and input space. We demonstrate its efficacy on several language modelling tasks.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑