Deep neural networks have achieved human-level accuracy on almost all perceptual benchmarks. It is interesting that these advances were made using two ideas that are decades old: (a) an artificial neuron based on a linear summator and (b) SGD training.
However, there are important metrics beyond accuracy: computational efficiency and stability against adversarial perturbations. In this paper, we propose two closely connected methods to improve these metrics on contour recognition tasks: (a) a novel model of an artificial neuron, a "strong neuron," with low hardware requirements and inherent robustness against adversarial perturbations and (b) a novel constructive training algorithm that generates sparse networks with $O(1)$ connections per neuron.
We demonstrate the feasibility of our approach through experiments on SVHN and GTSRB benchmarks. We achieved an impressive 10x-100x reduction in operations count (10x when compared with other sparsification approaches, 100x when compared with dense networks) and a substantial reduction in hardware requirements (8-bit fixed-point math was used) with no reduction in model accuracy. Superior stability against adversarial perturbations (exceeding that of adversarial training) was achieved without any counteradversarial measures, relying on the robustness of strong neurons alone. We also proved that constituent blocks of our strong neuron are the only activation functions with perfect stability against adversarial attacks.
Towards Interpretable Deep Neural Networks by Leveraging Adversarial Examples
Tsinghua National Lab for Information Science and Technology（清华大学信息科学与技术国家实验室）；State Key Lab of Intelligent Technology and Systems（智能技术与系统国家重点实验室）；Center for Bio-Inspired Computing Research（生物启发计算研究中心）；Department of Computer Science and Technology, Tsinghua University（清华大学计算机科学与技术系）
Fight Perturbations with Perturbations: Defending Adversarial Attacks via Neuron Influence
College of Information Engineering at Zhejiang University of Technology（浙江工业大学信息工程学院）；School of Cyber Science and Technology at Zhejiang University（浙江大学网络空间安全学院）
Adversarially Robust Spiking Neural Networks with Sparse Connectivity
Institute of Machine Learning and Neural Computation, Graz University of Technology, Austria（机器学习与神经计算研究所，格拉茨技术大学）；TU Graz - SAL Dependable Embedded Systems Lab, Silicon Austria Labs, Austria（格拉茨大学 - SAL 可靠嵌入式系统实验室，Silicon Austria Labs）；Chair of Cyber-Physical-Systems, Montanuniversität Leoben, Austria（智能物理系统系，蒙特安大学）
Adversarial dictionary learning for a robust analysis and modelling of spontaneous neuronal activity
University of Crete（克里特大学）；Foundation for Research and Technology Hellas（希腊研究与技术基金会）；Brigham and Women’s Hospital（布莱根妇女医院）；Harvard Medical School（哈佛医学院）；Boston VA Research Institute（波士顿退伍军人事务研究所）；Jamaica Plain Veterans Administration Hospital（牙买加平原退伍军人管理局医院）
Robust Spiking Neural Networks Against Adversarial Attacks
University of Electronic Science and Technology of China（电子科技大学）；Northumbria University（诺丁汉大学）；Shenzhen Loop Area Institute（深圳河套学院）；The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））
