Training neural networks sequentially in time to approximate solution fields of time-dependent partial differential equations can be beneficial for preserving causality and other physics properties; however, the sequential-in-time training is numerically challenging because training errors quickly accumulate and amplify over time. This work introduces Neural Galerkin schemes that update randomized sparse subsets of network parameters at each time step. The randomization avoids overfitting locally in time and so helps prevent the error from accumulating quickly over the sequential-in-time training, which is motivated by dropout that addresses a similar issue of overfitting due to neuron co-adaptation. The sparsity of the update reduces the computational costs of training without losing expressiveness because many of the network parameters are redundant locally at each time step. In numerical experiments with a wide range of evolution equations, the proposed scheme with randomized sparse updates is up to two orders of magnitude more accurate at a fixed computational budget and up to two orders of magnitude faster at a fixed accuracy than schemes with dense updates.
Theory of Dual-sparse Regularized Randomized Reduction
the University of Iowa（爱荷华大学）；Nanjing University（南京大学）；Michigan State University（密歇根州立大学）；Institute of Data Science and Technologies at Alibaba Group（阿里巴巴集团数据科学与技术研究院）
Domain-Decomposed Randomized Neural Networks for Partial Differential Equations in Unbounded Domains
School of Mathematics and Statistics, Xi’an Jiaotong University（西安交通大学数学与统计学院）；State Key Laboratory of Multiphase Flow in Power Engineering, Xi’an Jiaotong University（西安交通大学动力工程多相流国家重点实验室）
International Computer Science Institute（国际计算机科学研究所）；University of California Berkeley（加州大学伯克利分校）；Xerox Research Centre India（施乐印度研究中心）；Purdue University（普渡大学）
Robustness Certificates for Sparse Adversarial Attacks by Randomized Ablation
University of Maryland, College Park（马里兰大学帕克分校）
最近，技术已经发展到可证明地保证分类器的鲁棒性有界L_1和L_2幅度的对抗性扰动通过使用随机平滑：鲁棒分类是随机噪声样本的基础分类的共识，其中噪声是加性的。在本文中，我们将这种技术扩展到L_0威胁模型。我们提出了一个有效的和可证明的强大的防御稀疏对抗攻击随机消融输入功能，而不是使用加性噪声。实验上，在MNIST上，我们可以证明超过50%的图像的分类对最多8个像素的任何失真都是鲁棒的。这与MNIST上未受保护的分类器对现代L_0攻击的经验鲁棒性相当，证明了所提出的鲁棒性证书的紧密性。我们还评估了我们在ImageNet和CIFAR-10上的证书。我们的证书代表了对使用随机噪声而不是消融的并行工作（Lee et al. 2019）中提供的证书的改进（MNIST上的中值证书为8像素与4像素; ImageNet上的中值证书为16像素与1像素）。此外，我们的经验表明，我们的分类是高度稳健的现代稀疏对抗攻击MNIST。我们的分类在中位数上对高达31个像素的对抗性扰动是鲁棒的，相比之下，22个像素被报道为最先进的防御，其代价是分类准确性略有下降（约2.3%）。代码可在https://github.com/alevine0/randomizedAblation/上获得。
Department of Mathematics（数学系）；Hunan University（湖南大学）；Academy for Multidisciplinary Studies（跨学科研究学院）；Beijing National Center for Applied Mathematics（北京应用数学中心）；Capital Normal University（首都师范大学）
Learning Long-Term Reward Redistribution via Randomized Return Decomposition
University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；Shanghai Jiao Tong University（上海交通大学）；BIMSA（北京雁栖湖应用数学研究院）；AIR, Tsinghua University（清华大学智能产业研究院）；HeliXon Limited（华深智药）
