The biologically inspired spiking neurons used in neuromorphic computing are nonlinear filters with dynamic state variables -- very different from the stateless neuron models used in deep learning. The next version of Intel's neuromorphic research processor, Loihi 2, supports a wide range of stateful spiking neuron models with fully programmable dynamics. Here we showcase advanced spiking neuron models that can be used to efficiently process streaming data in simulation experiments on emulated Loihi 2 hardware. In one example, Resonate-and-Fire (RF) neurons are used to compute the Short Time Fourier Transform (STFT) with similar computational complexity but 47x less output bandwidth than the conventional STFT. In another example, we describe an algorithm for optical flow estimation using spatiotemporal RF neurons that requires over 90x fewer operations than a conventional DNN-based solution. We also demonstrate promising preliminary results using backpropagation to train RF neurons for audio classification tasks. Finally, we show that a cascade of Hopf resonators - a variant of the RF neuron - replicates novel properties of the cochlea and motivates an efficient spike-based spectrogram encoder.
Neuromorphic hardware is based on emulating the natural biological structure of the brain. Since its computational model is similar to standard neural models, it could serve as a computational acceleration for research projects in the field of neuroscience and artificial intelligence, including biomedical applications. However, in order to exploit this new generation of computer chips, rigorous simulation and consequent validation of brain-based experimental data is imperative. In this work, we investigate the potential of Intel's fifth generation neuromorphic chip - `Loihi', which is based on…
Spiking Neural Networks (SNNs) are a promising paradigm for efficient event-driven processing of spatio-temporally sparse data streams. SNNs have inspired the design and can take advantage of the emerging class of neuromorphic processors like Intel Loihi. These novel hardware architectures expose a variety of constraints that affect firmware, compiler and algorithm development alike. To enable rapid and flexible development of SNN algorithms on Loihi, we developed NxTF: a programming interface derived from Keras and compiler optimized for mapping deep convolutional SNNs to the multi-core Intel…
Comparing Quantum Annealing and Spiking Neuromorphic Computing for Sampling Binary Sparse Coding QUBO Problems
Los Alamos National Laboratory, CCS-3 Information Sciences（洛斯阿拉莫斯国家实验室，CCS-3信息科学部门）；Harvard University, T.H. Chan School of Public Health（哈佛大学，T.H. Chan公共卫生学院）
Although widely used in machine learning, backpropagation cannot directly be applied to SNN training and is not feasible on a neuromorphic processor that emulates biological neuron and synapses. This work presents a spike-based backpropagation algorithm with biological plausible local update rules and adapts it to fit the constraint in a neuromorphic hardware. The algorithm is implemented on Intel Loihi chip enabling low power in-hardware supervised online learning of multilayered SNNs for mobile applications. We test this implementation on MNIST, Fashion-MNIST, CIFAR-10 and MSTAR datasets wit…
Graz University of Technology（格拉茨技术大学）；Intel Corporation（英特尔公司）
Spike-based neuromorphic hardware holds the promise to provide more energy efficient implementations of Deep Neural Networks (DNNs) than standard hardware such as GPUs. But this requires to understand how DNNs can be emulated in an event-based sparse firing regime, since otherwise the energy-advantage gets lost. In particular, DNNs that solve sequence processing tasks typically employ Long Short-Term Memory (LSTM) units that are hard to emulate with few spikes. We show that a facet of many biological neurons, slow after-hyperpolarizing (AHP) currents after each spike, provides an efficient sol…
Temporal Neural Networks (TNNs) are spiking neural networks that exhibit brain-like sensory processing with high energy efficiency. This work presents the ongoing research towards developing a custom design framework for designing efficient application-specific TNN-based Neuromorphic Sensory Processing Units (NSPUs). This paper examines previous works on NSPU designs for UCR time-series clustering and MNIST image classification applications. Current ideas for a custom design framework and tools that enable efficient software-to-hardware design flow for rapid design space exploration of applica…
Neuromorphic hardware platforms can significantly lower the energy overhead of a machine learning inference task. We present a design-technology tradeoff analysis to implement such inference tasks on the processing elements (PEs) of a Non- Volatile Memory (NVM)-based neuromorphic hardware. Through detailed circuit-level simulations at scaled process technology nodes, we show the negative impact of technology scaling on the information-processing latency, which impacts the quality-of-service (QoS) of an embedded ML system. At a finer granularity, the latency inside a PE depends on 1) the delay …
Temporal Neural Networks (TNNs) use time as a resource to represent and process information, mimicking the behavior of the mammalian neocortex. This work focuses on implementing TNNs using off-the-shelf digital CMOS technology. A microarchitecture framework is introduced with a hierarchy of building blocks including: multi-neuron columns, multi-column layers, and multi-layer TNNs. We present the direct CMOS gate-level implementation of the multi-neuron column model as the key building block for TNNs. Post-synthesis results are obtained using Synopsys tools and the 45 nm CMOS standard cell libr…
School of EEE Nanyang Technological University（南洋理工大学电气与电子工程学院）
In this paper, we review recent work published over the last 3 years under the umbrella of Neuromorphic engineering to analyze what are the common features among such systems. We see that there is no clear consensus but each system has one or more of the following features:(1) Analog computing (2) Non vonNeumann Architecture and low-precision digital processing (3) Spiking Neural Networks (SNN) with components closely related to biology. We compare recent machine learning accelerator chips to show that indeed analog processing and reduced bit precision architectures have best throughput, energ…
Bottom-up and top-down approaches for the design of neuromorphic processing systems: Tradeoffs and synergies between natural and artificial intelligence
Institute of Neuroinformatics, University of Zurich and ETH Zurich（苏黎世大学与苏黎世联邦理工学院神经信息学研究所）；Delft University of Technology（代尔夫特理工大学）；ICTEAM Institute, Université catholique de Louvain（鲁汶天主教大学ICTEAM研究所）
In this work, we introduce an optoelectronic spiking artificial neuron capable of operating at ultrafast rates ($\approx$ 100 ps/optical spike) and with low energy consumption ($<$ pJ/spike). The proposed system combines an excitable resonant tunnelling diode (RTD) element exhibiting negative differential conductance, coupled to a nanoscale light source (forming a master node) or a photodetector (forming a receiver node). We study numerically the spiking dynamical responses and information propagation functionality of an interconnected master-receiver RTD node system. Using the key functionali…
The stringent memory and power constraints required in edge-computing sensory-processing applications have made event-driven neuromorphic systems a promising technology. On-chip online learning provides such systems the ability to learn the statistics of the incoming data and to adapt to their changes. Implementing online learning on event driven-neuromorphic systems requires (i) a spike-based learning algorithm that calculates the weight updates using only local information from streaming data, (ii) mapping these weight updates onto limited bit precision memory and (iii) doing so in a robust …
The ever-increasing demand for Artificial Intelligence (AI) systems is underlining a significant requirement for new, AI-optimised hardware. Neuromorphic (brain-like) processors are one highly-promising solution, with photonic-enabled realizations receiving increasing attention. Among these, approaches based upon Vertical Cavity Surface Emitting Lasers (VCSELs) are attracting interest given their favourable attributes and mature technology. Here, we demonstrate a hardware-friendly neuromorphic photonic spike processor, using a single VCSEL, for all-optical image edge-feature detection. This ex…
Bielefeld University（比勒费尔德大学）；University of Zurich（苏黎世大学）；ETH Zurich（苏黎世联邦理工学院）
The development of memristive device technologies has reached a level of maturity to enable the design of complex and large-scale hybrid memristive-CMOS neural processing systems. These systems offer promising solutions for implementing novel in-memory computing architectures for machine learning and data analysis problems. We argue that they are also ideal building blocks for the integration in neuromorphic electronic circuits suitable for ultra-low power brain-inspired sensory processing systems, therefore leading to the innovative solutions for always-on edge-computing and Internet-of-Thing…
Spiking Neural Networks (SNNs) provide an efficient computational mechanism for temporal signal processing, especially when coupled with low-power SNN inference ASICs. SNNs have been historically difficult to configure, lacking a general method for finding solutions for arbitrary tasks. In recent years, gradient-descent optimization methods have been applied to SNNs with increasing ease. SNNs and SNN inference processors therefore offer a good platform for commercial low-power signal processing in energy constrained environments without cloud dependencies. However, to date these methods have n…
Neuromorphic chip refers to an unconventional computing architecture that is modelled on biological brains. It is ideally suited for processing sensory data for intelligence computing, decision-making or context cognition. Despite rapid development, conventional artificial synapses exhibit poor connection flexibility and require separate data acquisition circuitry, resulting in limited functionalities and significant hardware redundancy. Here we report a novel light-stimulated artificial synapse based on a graphene-nanotube hybrid phototransistor that can directly convert optical stimuli into …
The marriage of two vibrant fields---photonics and neuromorphic processing---is fundamentally enabled by the strong analogies within the underlying physics between the dynamics of biological neurons and lasers, both of which can be understood within the framework of nonlinear dynamical systems theory. Whereas neuromorphic engineering exploits the biophysics of neuronal computation algorithms to provide a wide range of computing and signal processing applications, photonics offer an alternative approach to neuromorphic systems by exploiting the high speed, high bandwidth, and low crosstalk avai…
Photonic technologies offer great prospects for novel ultrafast, energy-efficient and hardware-friendly neuromorphic (brain-like) computing platforms. Moreover, neuromorphic photonic approaches based upon ubiquitous, technology-mature and low-cost Vertical-Cavity Surface Emitting Lasers (VCSELs) (devices found in fibre-optic transmitters, mobile phones, automotive sensors, etc.) are of particular interest. Given VCSELs have shown the ability to realise neuronal optical spiking responses (at ultrafast GHz rates), their use for spike-based information processing systems has been proposed. In thi…
Neuromorphic Computing implemented in photonic hardware is one of the most promising routes towards achieving machine learning processing at the picosecond scale, with minimum power consumption. In this work, we present a new concept for realizing photonic recurrent neural networks and reservoir computing architectures with the use of recurrent optical spectrum slicing. This is accomplished through simple optical filters placed in an loop, where each filter processes a specific spectral slice of the incoming optical signal. The synaptic weights in our scheme are equivalent to filters central f…
Vertical-Cavity Surface-Emitting Lasers (VCSELs) are highly promising devices for the construction of neuromorphic photonic information processing systems, due to their numerous desirable properties such as low power consumption, high modulation speed, compactness, and ease of manufacturing. Of particular interest is the ability of VCSELs to exhibit neural-like spiking responses, much like biological neurons, but at ultrafast sub-nanosecond rates; thus offering great prospects for high-speed light-enabled neuromorphic (spike-based) processors. Recent works have shown the use the spiking dynami…
