arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2410.04764cs.LGcs.GT
收藏

面向博弈论深度学习模型的双神谕神经网络架构搜索

Double Oracle Neural Architecture Search for Game Theoretic Deep Learning Models

Institute for Infocomm Research, Agency for Science, Technology and Research (A*STAR)(新加坡科技研究局信息通信研究院);School of Computer Science and Engineering, Nanyang Technological University(南洋理工大学计算机科学与工程学院);KTH Royal Institute of Technology(瑞典皇家理工学院);University of Nebraska-Lincoln(内布拉斯加大学林肯分校);A*STAR Centre for Frontier AI Research(新加坡科技研究局前沿人工智能研究中心)

Aye Phyu Phyu Aung, Xinrun Wang, Ruiyu Wang, Hau Chan, Bo An, Xiaoli Li, J. Senthilnath

AI总结:

该研究提出基于双神谕框架的博弈论深度学习训练方法,将其应用于GAN和AT的神经架构搜索,通过策略泛化、线性规划求元策略与弱占优策略剪枝保障效果与扩展性,实验验证其在多数据集上的性能与鲁棒性均优于基础架构。

摘要:

本文提出一种利用博弈论概念训练深度学习模型的新方法,涉及生成对抗网络(Generative Adversarial Networks, GANs)和对抗训练(Adversarial Training, AT),该方法采用基于最优响应神谕的双神谕框架。GAN本质上是生成器与判别器之间的双人零和博弈,同一概念也可应用于以攻击者和分类器为博弈方的AT。这类模型的训练颇具挑战:纯纳什均衡可能不存在,而即便寻找混合纳什均衡也十分困难,因为GAN和AT的训练算法都具备大规模策略空间。在我们的前期模型DO-GAN的基础上,本文提出将双神谕框架概念应用于对抗神经网络架构搜索(面向GAN的NAS)和对抗训练(面向AT的NAS)算法的方法。我们首先将博弈方策略泛化为来自最优响应神谕的生成器与判别器训练模型,随后通过线性规划计算元策略。针对框架中需在内存中存储多个最优响应网络模型的可扩展性问题,我们对弱占优的博弈方策略进行剪枝,避免神谕变得难以处理。最后,我们在MNIST、CIFAR-10和TinyImageNet数据集上对DONAS-GAN进行了实验,同时在CIFAR-10、SVHN和TinyImageNet数据集上评估了DONAS-AT在FGSM和PGD攻击下的鲁棒性。结果表明,与各自的基础架构相比,我们提出的所有变体在主观定性评估和定量指标上均有显著提升。

英文摘要:

In this paper, we propose a new approach to train deep learning models using game theory concepts including Generative Adversarial Networks (GANs) and Adversarial Training (AT) where we deploy a double-oracle framework using best response oracles. GAN is essentially a two-player zero-sum game between the generator and the discriminator. The same concept can be applied to AT with attacker and classifier as players. Training these models is challenging as a pure Nash equilibrium may not exist and even finding the mixed Nash equilibrium is difficult as training algorithms for both GAN and AT have a large-scale strategy space. Extending our preliminary model DO-GAN, we propose the methods to apply the double oracle framework concept to Adversarial Neural Architecture Search (NAS for GAN) and Adversarial Training (NAS for AT) algorithms. We first generalize the players' strategies as the trained models of generator and discriminator from the best response oracles. We then compute the meta-strategies using a linear program. For scalability of the framework where multiple network models of best responses are stored in the memory, we prune the weakly-dominated players' strategies to keep the oracles from becoming intractable. Finally, we conduct experiments on MNIST, CIFAR-10 and TinyImageNet for DONAS-GAN. We also evaluate the robustness under FGSM and PGD attacks on CIFAR-10, SVHN and TinyImageNet for DONAS-AT. We show that all our variants have significant improvements in both subjective qualitative evaluation and quantitative metrics, compared with their respective base architectures.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑