Developing mixed-signal analog-digital neuromorphic circuits in advanced scaled processes poses significant design challenges. We present compact and energy efficient sub-threshold analog synapse and neuron circuits, optimized for a 28 nm FD-SOI process, to implement massively parallel large-scale neuromorphic computing systems. We describe the techniques used for maximizing density with mixed-mode analog/digital synaptic weight configurations, and the methods adopted for minimizing the effect of channel leakage current, in order to implement efficient analog computation based on pA-nA small currents. We present circuit simulation results, based on a new chip that has been recently taped out, to demonstrate how the circuits can be useful for both low-frequency operation in systems that need to interact with the environment in real-time, and for high-frequency operation for fast data processing in different types of spiking neural network architectures.
As processes continue to scale aggressively, the design of deep sub-micron, mixed-signal design is becoming more and more challenging. In this paper we present an analysis of scaling multi-core mixed-signal neuromorphic processors to advanced 28 nm FD-SOI nodes. We address analog design issues which arise from the use of advanced process, including the problem of large leakage currents and device mismatch, and asynchronous digital design issues. We present the outcome of Monte Carlo Analysis and circuit simulations of neuromorphic sub threshold analog/digital neuron circuits which reproduce bi…
The first report on ion transport through atomic sieves of atomically-thin 2D material is provided to solve critical limitations of electrochemical random-access memory (ECRAM) devices.
This paper presents an extension of the BrainScaleS accelerated analog neuromorphic hardware model. The scalable neuromorphic architecture is extended by the support for multi-compartment models and non-linear dendrites. These features are part of a \SI{65}{\nano\meter} prototype ASIC. It allows to emulate different spike types observed in cortical pyramidal neurons: NMDA plateau potentials, calcium and sodium spikes. By replicating some of the structures of these cells, they can be configured to perform coincidence detection within a single neuron. Built-in plasticity mechanisms can modify no…
The progress in neuromorphic computing is fueled by the development of novel nonvolatile memories capable of storing analog information and implementing neural computation efficiently. However, like most other analog circuits, these devices and circuits are prone to imperfections, such as temperature dependency, noise, tuning error, etc., often leading to considerable performance degradation in neural network implementations. Indeed, imperfections are major obstacles in the path of further progress and ultimate commercialization of these technologies. Hence, a practically viable approach shoul…
Neuromorphic computing refers to brain-inspired computers, that differentiate it from von Neumann architecture. Analog VLSI based neuromorphic circuits is a current research interest. Two simpler spiking integrate and fire neuron model namely axon-Hillock (AH) and voltage integrate, and fire (VIF) circuits are commonly used for generating spike events. This paper discusses the impact of reliability issues like Bias Temperature instability (BTI) and Hot Carrier Injection (HCI), and timezero variability on these CMOS based neuromorphic circuits. AH and VIF circuits are implemented using HKMG bas…
Emerging analog resistive random access memory (RRAM) based on HfOx is an attractive device for non-von Neumann neuromorphic computing systems. The differences in temperature dependent conductance drift among cells hamper computing accuracy, characterized by the statistical distribution of temperature coefficient(Tα). A compact model was presented in order to investigate the statistical distribution of Tα under different resistance states. Based on this model, the physical mechanism of thermal instability of cells with a positive Tα was elucidated. Furthermore, this model can also effectively …
Neuromorphic systems typically employ current-mode circuits that model neural dynamics and produce output currents that range from few pico-Amperes to hundreds of micro-Amperes. On-line real-time monitoring of the signals produced by these circuits is crucial, for prototyping and debugging purposes, as well as for analyzing and understanding the network dynamics and computational properties. To this end, we propose a compact on-chip auto-scaling Current to Frequency Converter (CFC) for real-time monitoring of analog currents in mixed-signal/analog neuromorphic electronic systems. The proposed …
We experimentally demonstrate classification of 4x4 binary images into 4 classes, using a 3-layer mixed-signal neuromorphic network ("MLP perceptron"), based on two passive 20x20 memristive crossbar arrays, board-integrated with discrete CMOS components. The network features 10 hidden-layer and 4 output-layer analog CMOS neurons and 428 metal-oxide memristors, i.e. is almost an order of magnitude more complex than any previously reported functional memristor circuit. Moreover, the inference operation of this classifier is performed entirely in the integrated hardware. To deal with larger cross…
Heidelberg University（海德堡大学）；Friedrich Miescher Institute for Biomedical Research（弗里德里希·米歇尔生物医学研究所）
To rapidly process temporal information at a low metabolic cost, biological neurons integrate inputs as an analog sum but communicate with spikes, binary events in time. Analog neuromorphic hardware uses the same principles to emulate spiking neural networks with exceptional energy-efficiency. However, instantiating high-performing spiking networks on such hardware remains a significant challenge due to device mismatch and the lack of efficient training algorithms. Here, we introduce a general in-the-loop learning framework based on surrogate gradients that resolves these issues. Using the Bra…
We present a design-scheme for ultra-low power neuromorphic hardware using emerging spin-devices. We propose device models for 'neuron', based on lateral spin valves and domain wall magnets that can operate at ultra-low terminal voltage of ~20 mV, resulting in small computation energy. Magnetic tunnel junctions are employed for interfacing the spin-neurons with charge-based devices like CMOS, for large-scale networks. Device-circuit co-simulation-framework is used for simulating such hybrid designs, in order to evaluate system-level performance. We present the design of different classes of ne…
School of EEE Nanyang Technological University（南洋理工大学电气与电子工程学院）
In this paper, we review recent work published over the last 3 years under the umbrella of Neuromorphic engineering to analyze what are the common features among such systems. We see that there is no clear consensus but each system has one or more of the following features:(1) Analog computing (2) Non vonNeumann Architecture and low-precision digital processing (3) Spiking Neural Networks (SNN) with components closely related to biology. We compare recent machine learning accelerator chips to show that indeed analog processing and reduced bit precision architectures have best throughput, energ…
Nanoscale resistive switching devices (memristive devices or memristors) have been studied for a number of applications ranging from non-volatile memory, logic to neuromorphic systems. However a major challenge is to address the potentially large variations in space and in time in these nanoscale devices. Here we show that in metal-filament based memristive devices the switching can be fully stochastic. While individual switching events are random, the distribution and probability of switching can be well predicted and controlled. Rather than trying to force high switching probabilities using …
Several analog and digital brain-inspired electronic systems have been recently proposed as dedicated solutions for fast simulations of spiking neural networks. While these architectures are useful for exploring the computational properties of large-scale models of the nervous system, the challenge of building low-power compact physical artifacts that can behave intelligently in the real-world and exhibit cognitive abilities still remains open. In this paper we propose a set of neuromorphic engineering solutions to address this challenge. In particular, we review neuromorphic circuits for emul…
Neural coupled oscillators are a useful building block in numerous models and applications. They were analyzed extensively in theoretical studies and more recently, in biologically realistic simulations of spiking neural networks. The advent of mixed-signal analog/digital neuromorphic electronic circuits provides new means for implementing neural coupled oscillators on compact low-power spiking neural network hardware platforms. However, their implementation on this noisy, low-precision and inhomogeneous computing substrate raises new challenges with regards to stability and controllability. I…
The first-generation of BrainScaleS, also referred to as BrainScaleS-1, is a neuromorphic system for emulating large-scale networks of spiking neurons. Following a "physical modeling" principle, its VLSI circuits are designed to emulate the dynamics of biological examples: analog circuits implement neurons and synapses with time constants that arise from their electronic components' intrinsic properties. It operates in continuous time, with dynamics typically matching an acceleration factor of 10000 compared to the biological regime. A fault-tolerant design allows it to achieve wafer-scale int…
It has always been a challenge in the neuromorphic field to systematically translate biological models into analog electronic circuitry. In this paper, a generalized circuit design platform is introduced where biological models can be conveniently implemented using CMOS circuitry operating in strong-inversion. The application of the method is demonstrated by synthesizing a relatively complex two-dimensional (2-D) nonlinear neuron model. The validity of our approach is verified by nominal simulated results with realistic process parameters from the commercially available AMS 0.35 um technology.…
Dual-layer resistive switching devices with horizontal W electrodes, vertical Pd electrodes and WOx switching layer formed at the sidewall of the horizontal electrodes have been fabricated and characterized. The devices exhibit well-characterized analog switching characteristics and small mismatch in electrical characteristics for devices formed at the two layers. The three-dimensional (3D) vertical device structure allows higher storage density and larger connectivity for neuromorphic computing applications. We show the vertical devices exhibit potentiation and depression characteristics simi…
The approximation of quantum states with artificial neural networks has gained a lot of attention during the last years. Meanwhile, analog neuromorphic chips, inspired by structural and dynamical properties of the biological brain, show a high energy efficiency in running artificial neural-network architectures for the profit of generative applications. This encourages employing such hardware systems as platforms for simulations of quantum systems. Here we report on the realization of a prototype using the latest spike-based BrainScaleS hardware allowing us to represent few-qubit maximally ent…
Brain-inspired neuromorphic computing which consist neurons and synapses, with an ability to perform complex information processing has unfolded a new paradigm of computing to overcome the von Neumann bottleneck. Electronic synaptic memristor devices which can compete with the biological synapses are indeed significant for neuromorphic computing. In this work, we demonstrate our efforts to develop and realize the graphene oxide (GO) based memristor device as a synaptic device, which mimic as a biological synapse. Indeed, this device exhibits the essential synaptic learning behavior including a…
In an age overrun with information, the ability to process reams of data has become crucial. The demand for data will continue to grow as smart gadgets multiply and become increasingly integrated into our daily lives. Next-generation industries in artificial intelligence services and high-performance computing are so far supported by microelectronic platforms. These data-intensive enterprises rely on continual improvements in hardware. Their prospects are running up against a stark reality: conventional one-size-fits-all solutions offered by digital electronics can no longer satisfy this need,…
Jacobs University Bremen（不来梅雅各布大学）；University Medical Center Hamburg-Eppendorf（汉堡-埃彭多夫大学医学中心）
This chapter provides a comprehensive survey of the researches and motivations for hardware implementation of reservoir computing (RC) on neuromorphic electronic systems. Due to its computational efficiency and the fact that training amounts to a simple linear regression, both spiking and non-spiking implementations of reservoir computing on neuromorphic hardware have been developed. Here, a review of these experimental studies is provided to illustrate the progress in this area and to address the technical challenges which arise from this specific hardware implementation. Moreover, to deal wi…
We show analytically using a macrospin approximation that easy-plane spin Hall nano-oscillators excited by a spin-current polarized perpendicularly to the easy-plane have phase dynamics analogous to that of Josephson junctions. Similarly to Josephson junctions, they can reproduce the spiking behavior of biological neurons that is appropriate for neuromorphic computing. We perform micromagnetic simulations of such oscillators realized in the nano-constriction geometry and show that the easy-plane spiking dynamics is preserved in an experimentally feasible architecture. Finally we simulate two e…
Recent years have seen an increasing interest in the development of artificial intelligence circuits and systems for edge computing applications. In-memory computing mixed-signal neuromorphic architectures provide promising ultra-low-power solutions for edge-computing sensory-processing applications, thanks to their ability to emulate spiking neural networks in real-time. The fine-grain parallelism offered by this approach allows such neural circuits to process the sensory data efficiently by adapting their dynamics to the ones of the sensed signals, without having to resort to the time-multip…
The superior density of passive analog-grade memristive crossbars may enable storing large synaptic weight matrices directly on specialized neuromorphic chips, thus avoiding costly off-chip communication. To ensure efficient use of such crossbars in neuromorphic computing circuits, variations of current-voltage characteristics of crosspoint devices must be substantially lower than those of memory cells with select transistors. Apparently, this requirement explains why there were so few demonstrations of neuromorphic system prototypes using passive crossbars. Here we report a 64x64 passive meta…
Recent years have witnessed growing interest in the field of brain-inspired computing based on neural-network architectures. In order to translate the related algorithmic models into powerful, yet energy-efficient cognitive-computing hardware, computing-devices beyond CMOS may need to be explored. The suitability of such devices to this field of computing would strongly depend upon how closely their physical characteristics match with the essential computing primitives employed in such models. In this work we discuss the rationale of applying emerging spin-torque devices for bio-inspired compu…
A neuromorphic chip that combines CMOS analog spiking neurons and memristive synapses offers a promising solution to brain-inspired computing, as it can provide massive neural network parallelism and density. Previous hybrid analog CMOS-memristor approaches required extensive CMOS circuitry for training, and thus eliminated most of the density advantages gained by the adoption of memristor synapses. Further, they used different waveforms for pre and post-synaptic spikes that added undesirable circuit overhead. Here we describe a hardware architecture that can feature a large number of memristo…
Institute of Neuroinformatics（神经信息学研究所）；University of Zurich（苏黎世大学）；ETH Zurich（苏黎世联邦理工学院）
Efficient Balanced Networks (EBNs) are networks of spiking neurons in which excitatory and inhibitory synaptic currents are balanced on a short timescale, leading to desirable coding properties such as high encoding precision, low firing rates, and distributed information representation. It is for these benefits that it would be desirable to implement such networks in low-power neuromorphic processors. However, the degree of device mismatch in analog mixed-signal neuromorphic circuits renders the use of pre-trained EBNs challenging, if not impossible. To overcome this issue, we developed a nov…
Optical neural networks (ONNs), or optical neuromorphic hardware accelerators, have the potential to dramatically enhance the computing power and energy efficiency of mainstream electronic processors, due to their ultralarge bandwidths of up to 10s of terahertz together with their analog architecture that avoids the need for reading and writing data back and forth. Different multiplexing techniques have been employed to demonstrate ONNs, amongst which wavelength division multiplexing (WDM) techniques make sufficient use of the unique advantages of optics in terms of broad bandwidths. Here, we …
Memristor crossbar arrays are used in a wide range of in-memory and neuromorphic computing applications. However, memristor devices suffer from non-idealities that result in the variability of conductive states, making programming them to a desired analog conductance value extremely difficult as the device ages. In theory, memristors can be a nonlinear programmable analog resistor with memory properties that can take infinite resistive states. In practice, such memristors are hard to make, and in a crossbar, it is confined to a limited set of stable conductance values. The number of conductanc…
The first stage of tactile sensing is data acquisition using tactile sensors and the sensed data is transmitted to the central unit for neuromorphic computing. The memristive crossbars were proposed to use as synapses in neuromorphic computing but device intelligence at the sensor level are not investigated in literature. We propose the concept of Transistor Memristor Sensor (TMS)-crossbar by including sensor to memristor crossbar array configuration in the input layer of the neural network architecture. 2 possible cell configurations of TMS crossbar arrays: 1 Transistor 1 Memristor 1 Sensor (…
University of Twente（特文特大学）；Radboud University（拉德堡德大学）；IBM Research Europe - Zürich（IBM欧洲研究院-苏黎世）；Nikhef（荷兰国家亚原子物理研究所）；Eidgenössische Technische Hochschule Zürich（苏黎世联邦理工学院）；SURF Cooperation（SURF合作组织）；University of Valencia-CSIC（瓦伦西亚大学-西班牙国家研究委员会）
The massive use of artificial neural networks (ANNs), increasingly popular in many areas of scientific computing, rapidly increases the energy consumption of modern high-performance computing systems. An appealing and possibly more sustainable alternative is provided by novel neuromorphic paradigms, which directly implement ANNs in hardware. However, little is known about the actual benefits of running ANNs on neuromorphic hardware for use cases in scientific computing. Here we present a methodology for measuring the energy cost and compute time for inference tasks with ANNs on conventional ha…
The areal footprint of memristors is a key consideration in material-based neuromorophic computing and large-scale architecture integration. Electronic transport in the most widely investigated memristive devices is mediated by filaments, posing a challenge to their scalability in architecture implementation. Here we present a compelling alternative memristive device and demonstrate that areal downscaling leads to enhancement in memristive memory window, while maintaining analogue behavior, contrary to expectations. Our device designs directly integrated on semiconducting Nb-SrTiO$_3$ allows l…
We present an array of leaky integrate-and-fire (LIF) neuron circuits designed for the second-generation BrainScaleS mixed-signal 65-nm CMOS neuromorphic hardware. The neuronal array is embedded in the analog network core of a scaled-down prototype HICANN-DLS chip. Designed as continuous-time circuits, the neurons are highly tunable and reconfigurable elements with accelerated dynamics. Each neuron integrates input current from a multitude of incoming synapses and evokes a digital spike event output. The circuit offers a wide tuning range for synaptic and membrane time constants, as well as fo…
