The areal footprint of memristors is a key consideration in material-based neuromorophic computing and large-scale architecture integration. Electronic transport in the most widely investigated memristive devices is mediated by filaments, posing a challenge to their scalability in architecture implementation. Here we present a compelling alternative memristive device and demonstrate that areal downscaling leads to enhancement in memristive memory window, while maintaining analogue behavior, contrary to expectations. Our device designs directly integrated on semiconducting Nb-SrTiO$_3$ allows leveraging electric field effects at edges, increasing the dynamic range in smaller devices. Our findings are substantiated by studying the microscopic nature of switching using scanning transmission electron microscopy, in different resistive states, revealing an interfacial layer whose physical extent is influenced by applied electric fields. The ability of Nb-SrTiO$_3$ memristors to satisfy hardware and software requirements with downscaling, while significantly enhancing memristive functionalities, makes them strong contenders for non-von Neumann computing, beyond CMOS.
Neuromorphic computing based on spiking neural networks has the potential to significantly improve on-line learning capabilities and energy efficiency of artificial intelligence, specially for edge computing. Recent progress in computational neuroscience have demonstrated the importance of heterosynaptic plasticity for network activity regulation and memorization. Implementing heterosynaptic plasticity in hardware is thus highly desirable, but important materials and engineering challenges remain, calling for breakthroughs in neuromorphic devices. In this mini-review, we propose an overview of…
Bielefeld University（比勒费尔德大学）；University of Zurich（苏黎世大学）；ETH Zurich（苏黎世联邦理工学院）
The development of memristive device technologies has reached a level of maturity to enable the design of complex and large-scale hybrid memristive-CMOS neural processing systems. These systems offer promising solutions for implementing novel in-memory computing architectures for machine learning and data analysis problems. We argue that they are also ideal building blocks for the integration in neuromorphic electronic circuits suitable for ultra-low power brain-inspired sensory processing systems, therefore leading to the innovative solutions for always-on edge-computing and Internet-of-Thing…
In this position paper, we present a discussion on neuromorphic computing and especially the learning/training algorithm to design a series of brains with different memristive values to solve complex ill-posed inverse problems based on a Finite Element(FE) method. First, the neuromorphic computing is addressed and we focus on a type of memristive circuit computing that falls into the scope of neuromorphic computing. Secondly based on reference [1] in which the complex dynamics of the complex memristive circuit was studied, we design a method and an approach to train the memristive circuit so t…
In this paper, we propose an efficient predefined structured sparsity-based ex-situ training framework for a hybrid CMOS-memristive neuromorphic hardware for deep neural network to significantly lower the power consumption and computational complexity and improve scalability. The structure is verified on a wide range of datasets including MNIST handwritten recognition, breast cancer prediction, and mobile health monitoring. The results of this study show that compared to its fully connected version, the proposed structure provides significant power reduction while maintaining high classificati…
Neuromorphic computing has emerged as a promising avenue towards building the next generation of intelligent computing systems. It has been proposed that memristive devices, which exhibit history-dependent conductivity modulation, could efficiently represent the synaptic weights in artificial neural networks. However, precise modulation of the device conductance over a wide dynamic range, necessary to maintain high network accuracy, is proving to be challenging. To address this, we present a multi-memristive synaptic architecture with an efficient global counter-based arbitration scheme. We fo…
Nanoscale resistive switching devices (memristive devices or memristors) have been studied for a number of applications ranging from non-volatile memory, logic to neuromorphic systems. However a major challenge is to address the potentially large variations in space and in time in these nanoscale devices. Here we show that in metal-filament based memristive devices the switching can be fully stochastic. While individual switching events are random, the distribution and probability of switching can be well predicted and controlled. Rather than trying to force high switching probabilities using …
At the Faraday Discussion, in the paper titled `Neuromorphic computation with spiking memristors: habituation, experimental instantiation of logic gates and a novel sequence-sensitive perceptron model' it was demonstrated that a large amount of computation could be done in a sequential way using memristor current spikes (d.c. response). As these spikes are found in many memristors (possibly all), this novel approach could be highly useful for fast and reproducible memristor circuits. However, questions were raised as to whether these spikes were actually due to memristance or merely capacitanc…
Dual-layer resistive switching devices with horizontal W electrodes, vertical Pd electrodes and WOx switching layer formed at the sidewall of the horizontal electrodes have been fabricated and characterized. The devices exhibit well-characterized analog switching characteristics and small mismatch in electrical characteristics for devices formed at the two layers. The three-dimensional (3D) vertical device structure allows higher storage density and larger connectivity for neuromorphic computing applications. We show the vertical devices exhibit potentiation and depression characteristics simi…
Spike-based learning with memristive devices in neuromorphic computing architectures typically uses learning circuits that require overlapping pulses from pre- and post-synaptic nodes. This imposes severe constraints on the length of the pulses transmitted in the network, and on the network's throughput. Furthermore, most of these circuits do not decouple the currents flowing through memristive devices from the one stimulating the target neuron. This can be a problem when using devices with high conductance values, because of the resulting large currents. In this paper we propose a novel circu…
Memristive devices represent a promising technology for building neuromorphic electronic systems. In addition to their compactness and non-volatility features, they are characterized by computationally relevant physical properties, such as state-dependence, non-linear conductance changes, and intrinsic variability in both their switching threshold and conductance values, that make them ideal devices for emulating the bio-physics of real synapses. In this paper we present a spiking neural network architecture that supports the use of memristive devices as synaptic elements, and propose mixed-si…
The progress in the field of neural computation hinges on the use of hardware more efficient than the conventional microprocessors. Recent works have shown that mixed-signal integrated memristive circuits, especially their passive ('0T1R') variety, may increase the neuromorphic network performance dramatically, leaving far behind their digital counterparts. The major obstacle, however, is relatively immature memristor technology so that only limited functionality has been demonstrated to date. Here we experimentally demonstrate operation of one-hidden layer perceptron classifier entirely in th…
Memristive materials are related to neuromorphic applications as they can combine information processing with memory storage in a single computational element, just as biological neurons. Many of these bioinspired materials emulate the characteristics of memory and learning processes that happen in the brain. In this work, we report the memristive properties of a two-terminal (2-T) organic device based on ionic migration mediated by an ion-transport polymer. The material possesses unique memristive properties: it is reversibly switchable, shows tens of conductive states, presents Hebbian learn…
Implementation of Neuromorphic Systems using post Complementary Metal-Oxide-Semiconductor (CMOS) technology based Memristive Crossbar Array (MCA) has emerged as a promising solution to enable low-power acceleration of neural networks. However, the recent trend to design Deep Neural Networks (DNNs) for achieving human-like cognitive abilities poses significant challenges towards the scalable design of neuromorphic systems (due to the increase in computation/storage demands). Network pruning [7] is a powerful technique to remove redundant connections for designing optimally connected (maximally …
We experimentally demonstrate classification of 4x4 binary images into 4 classes, using a 3-layer mixed-signal neuromorphic network ("MLP perceptron"), based on two passive 20x20 memristive crossbar arrays, board-integrated with discrete CMOS components. The network features 10 hidden-layer and 4 output-layer analog CMOS neurons and 428 metal-oxide memristors, i.e. is almost an order of magnitude more complex than any previously reported functional memristor circuit. Moreover, the inference operation of this classifier is performed entirely in the integrated hardware. To deal with larger cross…
We present MEMprop, the adoption of gradient-based learning to train fully memristive spiking neural networks (MSNNs). Our approach harnesses intrinsic device dynamics to trigger naturally arising voltage spikes. These spikes emitted by memristive dynamics are analog in nature, and thus fully differentiable, which eliminates the need for surrogate gradient methods that are prevalent in the spiking neural network (SNN) literature. Memristive neural networks typically either integrate memristors as synapses that map offline-trained networks, or otherwise rely on associative learning mechanisms t…
Neuromorphic circuits mimic partial functionalities of brain in a bio-inspired information processing sense in order to achieve similar efficiencies as biological systems. While there are common mathematical models for neurons, which can be realized as nonlinear oscillating circuits, an electrical representation of the synaptic coupling between those is more challenging. Since this coupling strength depends on the learning procedure in the past, it should include a kind of memory. We believe that memristive devices, which are essentially nonlinear resistors with memory, are potential candidate…
Recent breakthroughs in neuromorphic computing show that local forms of gradient descent learning are compatible with Spiking Neural Networks (SNNs) and synaptic plasticity. Although SNNs can be scalably implemented using neuromorphic VLSI, an architecture that can learn using gradient-descent in situ is still missing. In this paper, we propose a local, gradient-based, error-triggered learning algorithm with online ternary weight updates. The proposed algorithm enables online training of multi-layer SNNs with memristive neuromorphic hardware showing a small loss in the performance compared wit…
The automated wafer inspection and quality control is a complex and time-consuming task, which can speed up using neuromorphic memristive architectures, as a separate inspection device or integrating directly into sensors. This paper presents the performance analysis and comparison of different neuromorphic architectures for patterned wafer quality inspection and classification. The application of non-volatile memristive devices in these architectures ensures low power consumption, small on-chip area scalability. We demonstrate that Long-Short Term Memory (LSTM) outperforms other architectures…
Measurement of the Crystallization and Phase Transition of Niobium Dioxide Thin-Films for Neuromorphic Computing Applications Using a Tube Furnace Optical Transmission System
Nazarbayev University（纳扎尔巴耶夫大学）；University of California Berkeley（加州大学伯克利分校）
The volume, veracity, variability, and velocity of data produced from the ever-increasing network of sensors connected to Internet pose challenges for power management, scalability, and sustainability of cloud computing infrastructure. Increasing the data processing capability of edge computing devices at lower power requirements can reduce several overheads for cloud computing solutions. This paper provides the review of neuromorphic CMOS-memristive architectures that can be integrated into edge computing devices. We discuss why the neuromorphic architectures are useful for edge devices and s…
Atomic Switch Networks (ASN) comprising silver iodide (AgI) junctions, a material previously unexplored as functional memristive elements within highly-interconnected nanowire networks, were employed as a neuromorphic substrate for physical Reservoir Computing (RC). This new class of ASN-based devices has been physically characterized and utilized to classify spoken digit audio data, demonstrating the utility of substrate-based device architectures where intrinsic material properties can be exploited to perform computation in-materio. This work demonstrates high accuracy in the classification …
The advances in the field of machine learning using neuromorphic systems have paved the pathway for extensive research on possibilities of hardware implementations of neural networks. Various memristive technologies such as oxide-based devices, spintronics and phase change materials have been explored to implement the core functional units of neuromorphic systems, namely the synaptic network, and the neuronal functionality, in a fast and energy efficient manner. However, various non-idealities in the crossbar implementations of the synaptic arrays can significantly degrade performance of neura…
The objective of this paper is to explore the possibility to couple two van der Pol (vdP) oscillators via a resistance-capacitance (RC) network comprising a Ag-TiOx-Al memristive device. The coupling was mediated by connecting the gate terminals of two programmable unijunction transistors (PUTs) through the network. In the high resistance state (HRS) the memresistance was in the order of MOhm leading to two independent selfsustained oscillators characterized by the different frequencies f1 and f2 and no phase relation between the oscillations. After a few cycles and in dependency of the mediat…
The emergence of nano-scale memristive devices encouraged many different research areas to exploit their use in multiple applications. One of the proposed applications was to implement synaptic connections in bio-inspired neuromorphic systems. Large-scale neuromorphic hardware platforms are being developed with increasing number of neurons and synapses, having a critical bottleneck in the online learning capabilities. Spike-timing-dependent plasticity (STDP) is a widely used learning mechanism inspired by biology which updates the synaptic weight as a function of the temporal correlation betwe…
In this paper, the hardware implementation of a neuromorphic system is presented. This system is composed of a Leaky Integrate-and-Fire with Latency (LIFL) neuron and a Spike-Timing Dependent Plasticity (STDP) synapse. LIFL neuron model allows to encode more information than the common Integrate-and-Fire models, typically considered for neuromorphic implementations. In our system LIFL neuron is implemented using CMOS circuits while memristor is used for the implementation of the STDP synapse. A description of the entire circuit is provided. Finally, the capabilities of the proposed architectur…
Despite all the progress of semiconductor integrated circuit technology, the extreme complexity of the human cerebral cortex makes the hardware implementation of neuromorphic networks with a comparable number of devices exceptionally challenging. One of the most prospective candidates to provide comparable complexity, while operating much faster and with manageable power dissipation, are so-called CrossNets based on hybrid CMOS/memristor circuits. In these circuits, the usual CMOS stack is augmented with one or several crossbar layers, with adjustable two-terminal memristors at each crosspoint…
Neuromorphic computing using spike-based learning has broad prospects in reducing computing power. Memristive neurons composed with two locally active memristors have been used to mimic the dynamical behaviors of biological neurons. In this work, the dynamic operating conditions of NbO2-based memristive neurons and their transformation boundaries between the spiking and the bursting are comprehensively investigated. Furthermore, the underlying mechanism of bursting is analyzed and the controllability of the number of spikes during each burst period is demonstrated. Finally, pattern classificat…
Neural processing on devices and circuits is fast becoming a popular approach to emulate biological neural networks. Elaborate CMOS and memristive technologies have been employed to achieve this, including chalcogenide-based in-memory computing concepts. Here we show that nano-scaled films of chalcogenide semiconductors can serve as building-blocks for novel types of neural computations where their tunable electronic and optical properties are jointly exploited. We demonstrate that ultrathin photoactive cavities of Ge-doped Selenide can emulate the computationally powerful non-linear operation…
Neuromorphic architectures offer great promise for achieving computation capacities beyond conventional Von Neumann machines. The essential elements for achieving this vision are highly scalable synaptic mimics that do not undermine biological fidelity. Here we demonstrate that single solid-state TiO2 memristors can exhibit non-associative plasticity phenomena observed in biological synapses, supported by their metastable memory state transition properties. We show that, contrary to conventional uses of solid-state memory, the existence of rate-limiting volatility is a key feature for capturin…
Memristive neural networks (MNNs), which use memristors as neurons or synapses, have become a hot research topic recently. However, most memristors are not compatible with mainstream integrated circuit technology and their stabilities in large-scale are not very well so far. In this paper, a hardware friendly MNN circuit is introduced, in which the memristive characteristics are implemented by digital integrated circuit. Through this method, spike timing dependent plasticity (STDP) and unsupervised learning are realized. A weight sharing mechanism is proposed to bridge the gap of network scale…
Neuromorphic systems that learn and predict from streaming inputs hold significant promise in pervasive edge computing and its applications. In this paper, a neuromorphic system that processes spatio-temporal information on the edge is proposed. Algorithmically, the system is based on hierarchical temporal memory that inherently offers online learning, resiliency, and fault tolerance. Architecturally, it is a full custom mixed-signal design with an underlying digital communication scheme and analog computational modules. Therefore, the proposed system features reconfigurability, real-time proc…
The potential of memristive devices is often seeing in implementing neuromorphic architectures for achieving brain-like computation. However, the designing procedures do not allow for extended manipulation of the material, unlike CMOS technology, the properties of the memristive material should be harnessed in the context of such computation, under the view that biological synapses are memristors. Here we demonstrate that single solid-state TiO2 memristors can exhibit associative plasticity phenomena observed in biological cortical synapses, and are captured by a phenomenological plasticity mo…
Emerging non-volatile memory (NVM) technologies offer unique advantages in energy efficiency, latency, and features such as computing-in-memory. Consequently, emerging NVM technologies are considered an ideal substrate for computation and storage in future-generation neuromorphic platforms. These technologies need to be evaluated for fundamental reliability and security issues. In this paper, we present \emph{NeuroHammer}, a security threat in ReRAM crossbars caused by thermal crosstalk between memory cells. We demonstrate that bit-flips can be deliberately induced in ReRAM devices in a crossb…
In a data-driven economy, virtually all industries benefit from advances in information technology -- powerful computing systems are critically important for rapid technological progress. However, this progress might be at risk of slowing down if we do not address the discrepancy between our current computing power demands and what the existing technologies can offer. Key limitations to improving energy efficiency are the excessive growth of data transfer costs associated with the von Neumann architecture and the fundamental limits of complementary metal-oxide-semiconductor (CMOS) technologies…
Logic-in-memory (LIM) describes the execution of logic gates within memristive crossbar structures, promising to improve performance and energy efficiency. Utilizing only binary values, LIM particularly excels in accelerating binary neural networks, shifting it in the focus of edge applications. Considering its potential, the impact of faults on BNNs accelerated with LIM still lacks investigation. In this paper, we propose faulty logic-in-memory (FLIM), a fault injection platform capable of executing full-fledged BNNs on LIM while injecting in-field faults. The results show that FLIM runs a si…
Spiking recurrent neural networks (RNNs) are a promising tool for solving a wide variety of complex cognitive and motor tasks, due to their rich temporal dynamics and sparse processing. However training spiking RNNs on dedicated neuromorphic hardware is still an open challenge. This is due mainly to the lack of local, hardware-friendly learning mechanisms that can solve the temporal credit assignment problem and ensure stable network dynamics, even when the weight resolution is limited. These challenges are further accentuated, if one resorts to using memristive devices for in-memory computing…
Brain-inspired computing proposes a set of algorithmic principles that hold promise for advancing artificial intelligence. They endow systems with self learning capabilities, efficient energy usage, and high storage capacity. A core concept that lies at the heart of brain computation is sequence learning and prediction. This form of computation is essential for almost all our daily tasks such as movement generation, perception, and language. Understanding how the brain performs such a computation is not only important to advance neuroscience but also to pave the way to new technological brain-…
Hierarchical Temporal Memory (HTM) is a neuromorphic algorithm that emulates sparsity, hierarchy and modularity resembling the working principles of neocortex. Feature encoding is an important step to create sparse binary patterns. This sparsity is introduced by the binary weights and random weight assignment in the initialization stage of the HTM. We propose the alternative deterministic method for the HTM initialization stage, which connects the HTM weights to the input data and preserves natural sparsity of the input information. Further, we introduce the hardware implementation of the dete…
The massive parallel approach of neuromorphic circuits leads to effective methods for solving complex problems. It has turned out that resistive switching devices with a continuous resistance range are potential candidates for such applications. These devices are memristive systems - nonlinear resistors with memory. They are fabricated in nanotechnology and hence parameter spread during fabrication may aggravate reproducible analyses. This issue makes simulation models of memristive devices worthwhile. Kinetic Monte-Carlo simulations based on a distributed model of the device can be used to un…
Throughout evolution the brain has mastered the art of processing real-world inputs through networks of interlinked spiking neurons. Synapses have emerged as key elements that, owing to their plasticity, are merging neuron-to-neuron signalling with memory storage and computation. Electronics has made important steps in emulating neurons through neuromorphic circuits and synapses with nanoscale memristors, yet novel applications that interlink them in heterogeneous bio-inspired and bio-hybrid architectures are just beginning to materialise. The use of memristive technologies in brain-inspired a…
