arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2302.01075stat.MLcs.LG

MonoFlow:从Wasserstein梯度流视角重新思考散度GAN

MonoFlow: Rethinking Divergence GANs via the Perspective of Wasserstein Gradient Flows

University of Bristol(布里斯托大学);Changping National Laboratory(昌平国家实验室);Peking University(北京大学)

Mingxuan Yi, Zhanxing Zhu, Song Liu

AI总结:

本文提出统一生成建模框架MonoFlow,利用Wasserstein梯度流重新解释GAN对抗训练,揭示变分散度最小化与对抗训练的本质区别,并指出只要实现MonoFlow,GAN可拥有更多损失设计。

摘要:

生成对抗网络(GANs)中对抗训练的传统理解是:判别器被训练来估计一个散度,生成器则学习最小化该散度。我们认为,尽管许多GAN变体都是遵循这一范式发展而来的,但当前对GANs的理论理解与其实际算法并不一致。在本文中,我们利用Wasserstein梯度流来刻画样本空间中粒子的演化,从而获得对GANs的理论洞见和算法启发。我们引入了一个统一的生成建模框架——MonoFlow:粒子演化通过对数密度比的单调递增映射进行重新缩放。在我们的框架下,对抗训练可以被视为一个过程:首先通过训练判别器获得MonoFlow的向量场,然后生成器学习绘制由相应向量场定义的粒子流。我们还揭示了变分散度最小化与对抗训练之间的根本区别。这一分析帮助我们识别哪些类型的生成器损失函数能够成功训练GANs,并表明只要能够实现MonoFlow,GANs可能拥有超出文献中现有设计(例如非饱和损失)的更多损失设计。我们包含了一致的实证研究来验证我们框架的有效性。

英文摘要:

The conventional understanding of adversarial training in generative adversarial networks (GANs) is that the discriminator is trained to estimate a divergence, and the generator learns to minimize this divergence. We argue that despite the fact that many variants of GANs were developed following this paradigm, the current theoretical understanding of GANs and their practical algorithms are inconsistent. In this paper, we leverage Wasserstein gradient flows which characterize the evolution of particles in the sample space, to gain theoretical insights and algorithmic inspiration of GANs. We introduce a unified generative modeling framework - MonoFlow: the particle evolution is rescaled via a monotonically increasing mapping of the log density ratio. Under our framework, adversarial training can be viewed as a procedure first obtaining MonoFlow's vector field via training the discriminator and the generator learns to draw the particle flow defined by the corresponding vector field. We also reveal the fundamental difference between variational divergence minimization and adversarial training. This analysis helps us to identify what types of generator loss functions can lead to the successful training of GANs and suggest that GANs may have more loss designs beyond the literature (e.g., non-saturated loss), as long as they realize MonoFlow. Consistent empirical studies are included to validate the effectiveness of our framework.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑