arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2305.07308cs.CV
收藏

通过多保真度评估高效搜索全面鲁棒的神经架构

Efficient Search of Comprehensively Robust Neural Architectures via Multi-fidelity Evaluation

Defense Innovation Institute, Chinese Academy of Military Science(中国人民解放军军事科学院国防科技创新研究院)

Jialiang Sun, Wen Yao, Tingsong Jiang, Xiaoqian Chen

AI总结:

针对现有NAS仅考虑$l_{\infty}$范数噪声导致鲁棒性不足的问题,提出ES-CRNA-ME方法,通过多类型评估、相关性分析及多保真度在线代理,高效搜索全面鲁棒架构,在CIFAR10和CIFAR100上验证有效性。

摘要:

神经架构搜索(NAS)已成为寻找鲁棒深度神经网络(DNN)架构的一项成功技术。然而,现有NAS中的大多数鲁棒性评估仅考虑基于$l_{\infty}$范数的对抗性噪声。为了提高DNN模型对多种类型噪声的鲁棒性,有必要在NAS中考虑对鲁棒架构进行全面评估。但随着鲁棒性评估类型数量的增加,寻找全面鲁棒架构也变得更加耗时。为缓解这一问题,我们提出了一种新颖的通过多保真度评估高效搜索全面鲁棒神经架构的方法(ES-CRNA-ME)。具体来说,我们首先使用基于权重共享的NAS方法,在多种评估类型下搜索全面鲁棒的架构,包括不同的$l_{p}$范数攻击、语义对抗攻击和复合对抗攻击。此外,我们通过相关性分析减少鲁棒性评估的数量,这可以合并相似的评估并降低评估成本。最后,我们在优化过程中提出了一种多保真度在线代理,以进一步降低搜索成本。在由低保真度数据构建的代理基础上,利用在线高保真度数据对代理进行微调。在CIFAR10和CIFAR100数据集上的实验证明了我们提出方法的有效性。

英文摘要:

Neural architecture search (NAS) has emerged as one successful technique to find robust deep neural network (DNN) architectures. However, most existing robustness evaluations in NAS only consider $l_{\infty}$ norm-based adversarial noises. In order to improve the robustness of DNN models against multiple types of noises, it is necessary to consider a comprehensive evaluation in NAS for robust architectures. But with the increasing number of types of robustness evaluations, it also becomes more time-consuming to find comprehensively robust architectures. To alleviate this problem, we propose a novel efficient search of comprehensively robust neural architectures via multi-fidelity evaluation (ES-CRNA-ME). Specifically, we first search for comprehensively robust architectures under multiple types of evaluations using the weight-sharing-based NAS method, including different $l_{p}$ norm attacks, semantic adversarial attacks, and composite adversarial attacks. In addition, we reduce the number of robustness evaluations by the correlation analysis, which can incorporate similar evaluations and decrease the evaluation cost. Finally, we propose a multi-fidelity online surrogate during optimization to further decrease the search cost. On the basis of the surrogate constructed by low-fidelity data, the online high-fidelity data is utilized to finetune the surrogate. Experiments on CIFAR10 and CIFAR100 datasets show the effectiveness of our proposed method.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑