arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2406.15152cs.LGcs.AIstat.ML
收藏

生成拓扑网络

Generative Topological Networks

Reichman University(赖希曼大学);Technion – Israel Institute of Technology(以色列理工学院);University of Oxford(牛津大学)

Alona Levy-Jurgenson, Zohar Yakhini

AI总结:

提出基于拓扑理论的生成拓扑网络(GTNs),采用监督学习在低维潜在空间生成数据,避免模式崩溃等缺陷,并解释低维生成提升质量的原因,实验证明优于VAEs且收敛快。

摘要:

生成方法近期通过在数据的低维潜在表示中进行生成而获得了显著改进。然而,许多应用于潜在空间的生成方法仍然复杂且难以训练。此外,为何过渡到低维潜在空间能够提升生成质量尚不完全清楚。在本工作中,我们引入了一种基于拓扑理论的新颖且简单的生成方法——生成拓扑网络(GTNs)——该方法也为低维潜在空间表示为何可能更适合数据生成提供了见解。GTNs易于训练——它们采用标准的监督学习方法,并且不会遭受常见的生成缺陷,如模式崩溃、后验崩溃或需要对神经网络架构施加约束。我们通过在数据的低维潜在表示上训练GTNs,在多个数据集上展示了其用途,包括MNIST、CelebA、CIFAR-10以及手部和手掌图像数据集。我们表明GTNs能够改进VAEs,并且它们收敛迅速,在早期epoch中即可生成逼真的样本。此外,我们利用GTNs开发背后的拓扑考量,为生成模型为何可能受益于在低维潜在空间上运行提供了见解,强调了数据的内在维度与数据生成维度之间的重要联系。特别地,我们证明在高维环境空间中生成可能是扩散模型生成分布外样本的一个促成因素。我们还强调了在使用和设计生成模型时需要考虑的其他重要拓扑属性。我们的代码可在以下网址获取:https://github.com/alonalj/GTN

英文摘要:

Generative methods have recently seen significant improvements by generating in a lower-dimensional latent representation of the data. However, many of the generative methods applied in the latent space remain complex and difficult to train. Further, it is not entirely clear why transitioning to a lower-dimensional latent space can improve generative quality. In this work, we introduce a new and simple generative method grounded in topology theory -- Generative Topological Networks (GTNs) -- which also provides insights into why lower-dimensional latent-space representations might be better-suited for data generation. GTNs are simple to train -- they employ a standard supervised learning approach and do not suffer from common generative pitfalls such as mode collapse, posterior collapse or the need to pose constraints on the neural network architecture. We demonstrate the use of GTNs on several datasets, including MNIST, CelebA, CIFAR-10 and the Hands and Palm Images dataset by training GTNs on a lower-dimensional latent representation of the data. We show that GTNs can improve upon VAEs and that they are quick to converge, generating realistic samples in early epochs. Further, we use the topological considerations behind the development of GTNs to offer insights into why generative models may benefit from operating on a lower-dimensional latent space, highlighting the important link between the intrinsic dimension of the data and the dimension in which the data is generated. Particularly, we demonstrate that generating in high dimensional ambient spaces may be a contributing factor to out-of-distribution samples generated by diffusion models. We also highlight other topological properties that are important to consider when using and designing generative models. Our code is available at: https://github.com/alonalj/GTN

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑