使用Lookahead-Minmax驯服GANs
Taming GANs with Lookahead-Minmax
摘要：
众所周知，生成对抗网络的训练具有挑战性。潜在的极小极大优化是非常容易受到随机梯度的方差和相关的游戏向量场的旋转分量。为了解决这些挑战，我们提出了前瞻算法的极大极小优化，最初开发的单目标最小化。我们的Lookahead-minmax的回溯步骤自然地处理旋转博弈动态，这一属性被认为是使梯度上升下降方法收敛于文献中经常分析的具有挑战性的示例的关键。此外，它隐式地处理高方差，而不使用大的小批量，这对于达到最先进的性能是必不可少的。在MNIST、SVHN、CIFAR-10和ImageNet上的实验结果表明，将Lookahead-minmax与Adam或extragradient相结合，在性能和稳定性方面具有明显的优势，内存和计算成本可以忽略不计。使用少30倍的参数和小16倍的小批量，我们通过在不使用类标签的情况下获得12.19的FID，超越了CIFAR-10上的类相关BigGAN的报告性能，从而使最先进的GAN训练在公共计算资源的范围内。
英文摘要：
Generative Adversarial Networks are notoriously challenging to train. The underlying minmax optimization is highly susceptible to the variance of the stochastic gradient and the rotational component of the associated game vector field. To tackle these challenges, we propose the Lookahead algorithm for minmax optimization, originally developed for single objective minimization only. The backtracking step of our Lookahead-minmax naturally handles the rotational game dynamics, a property which was identified to be key for enabling gradient ascent descent methods to converge on challenging examples often analyzed in the literature. Moreover, it implicitly handles high variance without using large mini-batches, known to be essential for reaching state of the art performance. Experimental results on MNIST, SVHN, CIFAR-10, and ImageNet demonstrate a clear advantage of combining Lookahead-minmax with Adam or extragradient, in terms of performance and improved stability, for negligible memory and computational cost. Using 30-fold fewer parameters and 16-fold smaller minibatches we outperform the reported performance of the class-dependent BigGAN on CIFAR-10 by obtaining FID of 12.19 without using the class labels, bringing state-of-the-art GAN training within reach of common computational resources.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
