Enhancing Reliability of Neural Networks at the Edge: Inverted Normalization with Stochastic Affine Transformations
Karlsruhe Institute of Technology(卡尔斯鲁厄理工学院);Univ. Grenoble Alpes, CEA, CNRS, Grenoble INP, and IRIG-Spintec(格勒诺布尔阿尔卑斯大学、法国原子能委员会、法国国家科学研究中心、格勒诺布尔国立理工学院及IRIG-Spintec)
Soyed Tuhin Ahmed, Kamal Danouchi, Guillaume Prenat, Lorena Anghel, Mehdi B. Tahoori
Bayesian Neural Networks (BayNNs) naturally provide uncertainty in their predictions, making them a suitable choice in safety-critical applications. Additionally, their realization using memristor-based in-memory computing (IMC) architectures enables them for resource-constrained edge applications. In addition to predictive uncertainty, however, the ability to be inherently robust to noise in computation is also essential to ensure functional safety. In particular, memristor-based IMCs are susceptible to various sources of non-idealities such as manufacturing and runtime variations, drift, and failure, which can significantly reduce inference accuracy. In this paper, we propose a method to inherently enhance the robustness and inference accuracy of BayNNs deployed in IMC architectures. To achieve this, we introduce a novel normalization layer combined with stochastic affine transformations. Empirical results in various benchmark datasets show a graceful degradation in inference accuracy, with an improvement of up to $58.11\%$.
AffineLens: Capturing the Continuous Piecewise Affine Functions of Neural Networks
State Key Laboratory of Novel Software Technology School of Intelligence Science and Technology Nanjing University, Jiangsu, China(软件新技术国家重点实验室 智能科学与技术学院 南京大学 江苏 中国);AI for Good Istituto Italiano di Tecnologia, Genoa, Italy DITEN University of Genoa, Genoa, Italy(AI for Good 意大利技术研究院,热那亚,意大利 DITEN 热那亚大学,热那亚,意大利);State Key Laboratory of Novel Software Technology School of Artificial Intelligence Nanjing University, Jiangsu, China(软件新技术国家重点实验室 人工智能学院 南京大学 江苏 中国);AI for Good Istituto Italiano di Tecnologia, Genoa, Italy Department of Computer Science University of Verona, Verona, Italy(AI for Good 意大利技术研究院,热那亚,意大利 计算机科学系 热那亚大学,热那亚,意大利)
Training-Time Batch Normalization Reshapes Local Partition Geometry in Piecewise-Affine Networks
AI for Good Istituto Italiano di Tecnologia(AI for Good 意大利技术研究院);DITEN University of Genoa(DITEN Genoa大学);State Key Laboratory of Novel Software Technology School of Intelligence Science and Technology(新型软件技术国家重点实验室 智能科学与技术学院);School of Artificial Intelligence Nanjing University(人工智能学院 南京大学);Department of Computer Science University of Verona(计算机科学系 Verona大学)
Parallel Layer Normalization for Universal Approximation
State Key Laboratory of Complex and Critical Software Environment (SKLCCSE)(复杂与关键软件环境国家重点实验室);Beihang University(北京航空航天大学);Hangzhou International Innovation Institute(杭州国际创新研究院)