我们研究了深度神经网络在（选定的）宽和窄，深和浅，以及懒惰和非懒惰训练设置中的平均鲁棒性概念。我们证明，在欠参数化设置，宽度有负面影响，而它提高了鲁棒性在过参数化设置。深度的效果与初始化和训练模式密切相关。特别是，当使用LeCun初始化时，深度有助于懒惰训练机制的鲁棒性。相反，当使用神经切线核（NTK）和He初始化进行初始化时，深度损害鲁棒性。此外，在非懒惰训练机制下，我们展示了双层ReLU网络的宽度如何有利于鲁棒性。我们的理论发展改进了[Huang et al. NeurIPS 21; Wu et al. NeurIPS 21]的结果，并与[Bubeck and Sellke NeurIPS 21; Bubeck et al. COLT 21]一致。
英文摘要：
We study the average robustness notion in deep neural networks in (selected) wide and narrow, deep and shallow, as well as lazy and non-lazy training settings. We prove that in the under-parameterized setting, width has a negative effect while it improves robustness in the over-parameterized setting. The effect of depth closely depends on the initialization and the training mode. In particular, when initialized with LeCun initialization, depth helps robustness with the lazy training regime. In contrast, when initialized with Neural Tangent Kernel (NTK) and He-initialization, depth hurts the robustness. Moreover, under the non-lazy training regime, we demonstrate how the width of a two-layer ReLU network benefits robustness. Our theoretical developments improve the results by [Huang et al. NeurIPS21; Wu et al. NeurIPS21] and are consistent with [Bubeck and Sellke NeurIPS21; Bubeck et al. COLT21].
Vertical Federated Learning in Practice: The Good, the Bad, and the Ugly
National University of Singapore（新加坡国立大学）；WeBank（微众银行）；Zhejiang University（浙江大学）；Shanghai Jiao Tong University（上海交通大学）；Huazhong University of Science and Technology（华中科技大学）
The Good, the Bad and the Ugly: Augmenting a black-box model with expert knowledge
Fraunhofer Center for Machine Learning（弗劳恩霍夫机器学习中心）；Fraunhofer Institute for Industrial Mathematics ITWM（弗劳恩霍夫工业数学研究所（ITWM））；Fraunhofer Institute for Mechanics of Materials IWM（弗劳恩霍夫材料力学研究所（IWM））
The Good, the Bad, and the Ugly: The Role of AI Quality Disclosure in Lie Detection
University of Utah（犹他大学）；Allen Institute for AI（艾伦人工智能研究所）；University of Reading（雷丁大学）；Indian Statistical Institute, Kolkata（印度统计研究所（加尔各答））；Centre for Studies in Social Sciences, Kolkata（社会科学研究中心（加尔各答））；Central European University（中欧大学）
Simulation, Modelling and Classification of Wiki Contributors: Spotting The Good, The Bad, and The Ugly
University of Vigo（维戈大学）；Universidade Portucalense（波尔图卡伦塞大学）；Polytechnic Institute of Porto（波尔图理工学院）；INESC TEC（INESC TEC研究院）；University of Porto（波尔图大学）；National College of Ireland（爱尔兰国立学院）
The Good, the Bad and the Ugly: Evaluating Convolutional Neural Networks for Prohibited Item Detection Using Real and Synthetically Composited X-ray Imagery
