arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2303.13813cs.CVcs.AIcs.LG
收藏

通才:解耦的自然和鲁棒的泛化

Generalist: Decoupling Natural and Robust Generalization

Peking University(北京大学);University of Hong Kong(香港大学)

Hongjun Wang, Yisen Wang

摘要:

通过标准训练获得的深度神经网络一直受到对抗性样本的困扰。尽管对抗性训练证明了它能够防御对抗性示例,但不幸的是,它导致了自然泛化能力的不可避免的下降。为了解决这个问题,我们将自然泛化和鲁棒泛化从联合训练中分离出来,并为每一个制定不同的训练策略。具体而言,而不是最大限度地减少这两个泛化误差的期望的全球损失,我们提出了一个双专家框架,称为\emph{Generalist},我们同时训练基地学习者的任务意识的战略,使他们能够专注于自己的领域。在训练过程中,每隔一段时间收集基本学习器的参数并组合形成全局学习器。然后将全局学习器作为初始化参数分发给基本学习器,以继续训练。理论上,我们证明了基础学习者经过良好的训练后,通才的风险会降低。大量的实验验证了Generalist的适用性,以实现高精度的自然例子,同时保持相当的鲁棒性对抗的。代码可在https://github.com/PKU-ML/Generalist上获得。

英文摘要:

Deep neural networks obtained by standard training have been constantly plagued by adversarial examples. Although adversarial training demonstrates its capability to defend against adversarial examples, unfortunately, it leads to an inevitable drop in the natural generalization. To address the issue, we decouple the natural generalization and the robust generalization from joint training and formulate different training strategies for each one. Specifically, instead of minimizing a global loss on the expectation over these two generalization errors, we propose a bi-expert framework called \emph{Generalist} where we simultaneously train base learners with task-aware strategies so that they can specialize in their own fields. The parameters of base learners are collected and combined to form a global learner at intervals during the training process. The global learner is then distributed to the base learners as initialized parameters for continued training. Theoretically, we prove that the risks of Generalist will get lower once the base learners are well trained. Extensive experiments verify the applicability of Generalist to achieve high accuracy on natural examples while maintaining considerable robustness to adversarial ones. Code is available at https://github.com/PKU-ML/Generalist.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑