arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2102.01886cs.LGcs.CV
收藏

学习多样结构网络的对抗鲁棒性

Learning Diverse-Structured Networks for Adversarial Robustness

Hong Kong Baptist University(香港浸会大学);University of Wisconsin-Madison(威斯康星大学麦迪逊分校);RIKEN(理化学研究所);University of Sydney(悉尼大学);Tencent AI Lab(腾讯人工智能实验室);University of Tokyo(东京大学)

Xuefeng Du, Jingfeng Zhang, Bo Han, Tongliang Liu, Yu Rong, Gang Niu, Junzhou Huang, Masashi Sugiyama

摘要:

在对抗训练(AT)中,主要关注的是目标和优化器,而模型研究较少,因此使用的模型仍然是标准训练(ST)中的经典模型。经典网络架构(NA)通常比ST中搜索的NA更差,AT中的NA应该是相同的。在本文中,我们认为NA和AT不能独立处理,因为给定一个数据集,ST中的最佳NA将不再是AT中的最佳NA。也就是说,AT本身是耗时的;如果我们在大的搜索空间上直接搜索AT中的NA,则计算实际上是不可行的。因此,我们提出了一种多样结构的网络(DS-Net),以显着减少搜索空间的大小:而不是低级别的操作,我们只考虑预定义的原子块,其中原子块是一个经过时间考验的构建块,如残留块。只有几个原子块,因此我们可以对所有原子块进行加权,而不是在DS-Net的搜索块中找到最好的一个,这是探索不同结构和利用最佳结构之间的必要权衡。实证结果证明了DS-Net的优势,即,对原子块进行加权。

英文摘要:

In adversarial training (AT), the main focus has been the objective and optimizer while the model has been less studied, so that the models being used are still those classic ones in standard training (ST). Classic network architectures (NAs) are generally worse than searched NAs in ST, which should be the same in AT. In this paper, we argue that NA and AT cannot be handled independently, since given a dataset, the optimal NA in ST would be no longer optimal in AT. That being said, AT is time-consuming itself; if we directly search NAs in AT over large search spaces, the computation will be practically infeasible. Thus, we propose a diverse-structured network (DS-Net), to significantly reduce the size of the search space: instead of low-level operations, we only consider predefined atomic blocks, where an atomic block is a time-tested building block like the residual block. There are only a few atomic blocks and thus we can weight all atomic blocks rather than find the best one in a searched block of DS-Net, which is an essential trade-off between exploring diverse structures and exploiting the best structures. Empirical results demonstrate the advantages of DS-Net, i.e., weighting the atomic blocks.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑