Sorting is needed in many application domains. The data is read from memory and sent to a general purpose processor or application specific hardware for sorting. The sorted data is then written back to the memory. Reading/writing data from/to memory and transferring data between memory and processing unit incur a large latency and energy overhead. In this work, we develop, to the best of our knowledge, the first architectures for in-memory sorting of data. We propose two architectures. The first architecture is applicable to the conventional format of representing data, weighted binary radix. The second architecture is proposed for the developing unary processing systems where data is encoded as uniform unary bitstreams. The two architectures have different advantages and disadvantages, making one or the other more suitable for a specific application. However, the common property of both is a significant reduction in the processing time compared to prior sorting designs. Our evaluations show on average 37x and 138x energy reduction for binary and unary designs, respectively, as compared to conventional CMOS off-memory sorting systems.
This paper presents a survey of the currently available hardware designs for implementation of the human cortex inspired algorithm, Hierarchical Temporal Memory (HTM). In this review, we focus on the state of the art advances of memristive HTM implementation and related HTM applications. With the advent of edge computing, HTM can be a potential algorithm to implement on-chip near sensor data processing. The comparison of analog memristive circuit implementations with the digital and mixed-signal solutions are provided. The advantages of memristive HTM over digital implementations against perfo…
Mapping neuro-inspired algorithms to sensor backplanes of on-chip hardware require shifting the signal processing from digital to the analog domain, demanding memory technologies beyond conventional CMOS binary storage units. Using memristors for building analog data storage is one of the promising approaches amongst emerging non-volatile memory technologies. Recently, a memristive multi-level memory (MLM) cell for storing discrete analog values has been developed in which memory system is implemented combining memristors in voltage divider configuration. In given example, the memory cell of 3…
Data-intensive applications are poised to benefit directly from processing-in-memory platforms, such as memristive Memory Processing Units, which allow leveraging data locality and performing stateful logic operations. Developing design automation flows for such platforms is a challenging and highly relevant research problem. In this work, we investigate the problem of minimizing delay under arbitrary area constraint for MAGIC-based in-memory computing platforms. We propose an end-to-end area constrained technology mapping framework, CONTRA. CONTRA uses Look-Up Table(LUT) based mapping of the …
In-memory computing is an emerging non-von Neumann computing paradigm where certain computational tasks are performed in memory by exploiting the physical attributes of the memory devices. Memristive devices such as phase-change memory (PCM), where information is stored in terms of their conductance levels, are especially well suited for in-memory computing. In particular, memristive devices, when organized in a crossbar configuration can be used to perform matrix-vector multiply operations by exploiting Kirchhoff's circuit laws. To explore the feasibility of such in-memory computing cores in …
CMOS technology and its continuous scaling have made electronics and computers accessible and affordable for almost everyone on the globe; in addition, they have enabled the solutions of a wide range of societal problems and applications. Today, however, both the technology and the computer architectures are facing severe challenges/walls making them incapable of providing the demanded computing power with tight constraints. This motivates the need for the exploration of novel architectures based on new device technologies; not only to sustain the financial benefit of technology scaling, but a…
The areal footprint of memristors is a key consideration in material-based neuromorophic computing and large-scale architecture integration. Electronic transport in the most widely investigated memristive devices is mediated by filaments, posing a challenge to their scalability in architecture implementation. Here we present a compelling alternative memristive device and demonstrate that areal downscaling leads to enhancement in memristive memory window, while maintaining analogue behavior, contrary to expectations. Our device designs directly integrated on semiconducting Nb-SrTiO$_3$ allows l…
Electrophysiological techniques have improved substantially over the past years to the point that neuroprosthetics applications are becoming viable. This evolution has been fuelled by the advancement of implantable microelectrode technologies that have followed their own version of Moore's scaling law. Similarly to electronics, however, excessive data-rates and strained power budgets require the development of more efficient computation paradigms for handling neural data in-situ, in particular the computationally heavy task of events classification. Here, we demonstrate how the intrinsic analo…
Hierarchical Temporal Memory is a new machine learning algorithm intended to mimic the working principle of neocortex, part of the human brain, which is responsible for learning, classification, and making predictions. Although many works illustrate its effectiveness as a software algorithm, hardware design for HTM remains an open research problem. Hence, this work proposes an architecture for HTM Spatial Pooler and Temporal Memory with learning mechanism, which creates a single image for each class based on important and unimportant features of all images in the training set. In turn, the red…
Hierarchical Temporal Memory (HTM) is a neuromorphic algorithm that emulates sparsity, hierarchy and modularity resembling the working principles of neocortex. Feature encoding is an important step to create sparse binary patterns. This sparsity is introduced by the binary weights and random weight assignment in the initialization stage of the HTM. We propose the alternative deterministic method for the HTM initialization stage, which connects the HTM weights to the input data and preserves natural sparsity of the input information. Further, we introduce the hardware implementation of the dete…
Memristive devices represent a promising technology for building neuromorphic electronic systems. In addition to their compactness and non-volatility features, they are characterized by computationally relevant physical properties, such as state-dependence, non-linear conductance changes, and intrinsic variability in both their switching threshold and conductance values, that make them ideal devices for emulating the bio-physics of real synapses. In this paper we present a spiking neural network architecture that supports the use of memristive devices as synaptic elements, and propose mixed-si…
Emerging non-volatile memory (NVM) technologies offer unique advantages in energy efficiency, latency, and features such as computing-in-memory. Consequently, emerging NVM technologies are considered an ideal substrate for computation and storage in future-generation neuromorphic platforms. These technologies need to be evaluated for fundamental reliability and security issues. In this paper, we present \emph{NeuroHammer}, a security threat in ReRAM crossbars caused by thermal crosstalk between memory cells. We demonstrate that bit-flips can be deliberately induced in ReRAM devices in a crossb…
Tree-based machine learning techniques, such as Decision Trees and Random Forests, are top performers in several domains as they do well with limited training datasets and offer improved interpretability compared to Deep Neural Networks (DNN). However, while easier to train, they are difficult to optimize for fast inference without accuracy loss in von Neumann architectures due to non-uniform memory access patterns. Recently, we proposed a novel analog, or multi-bit, content addressable memory(CAM) for fast look-up table operations. Here, we propose a design utilizing this as a computational p…
We show that memristive networks-namely networks of resistors with memory-can efficiently solve shortest-path optimization problems. Indeed, the presence of memory (time non-locality) promotes self organization of the network into the shortest possible path(s). We introduce a network entropy function to characterize the self-organized evolution, show the solution of the shortest-path problem and demonstrate the healing property of the solution path. Finally, we provide an algorithm to solve the traveling salesman problem. Similar considerations apply to networks of memcapacitors and meminducto…
The prediction made by L. O. Chua 45+ years ago (see: IEEE Trans. Circuit Theory (1971) 18:507-519 and also: Proc. IEEE (2012) 100:1920-1927) about the existence of a passive circuit element (called memristor) that links the charge and flux variables has been confirmed by the HP lab group in its report (see: Nature (2008) 453:80-83) on a successful construction of such an element. This sparked an enormous interest in mem-elements, analysis of their unusual dynamical properties (i.e. pinched hysteresis loops, memory effects, etc.) and construction of their emulators. Such topics are also of int…
The University of Hong Kong（香港大学）；Hewlett Packard Labs, Hewlett Packard Enterprise（慧与（HPE）惠普实验室）；University of Notre Dame（圣母大学）；Forschungszentrum Jülich GmbH（于利希研究中心）；RWTH Aachen University（亚琛工业大学）
Lifelong on-device learning is a key challenge for machine intelligence, and this requires learning from few, often single, samples. Memory augmented neural network has been proposed to achieve the goal, but the memory module has to be stored in an off-chip memory due to its size. Therefore the practical use has been heavily limited. Previous works on emerging memory-based implementation have difficulties in scaling up because different modules with various structures are difficult to integrate on the same chip and the small sense margin of the content addressable memory for the memory module …
Analog memory is of great importance in neurocomputing technologies field, but still remains difficult to implement. With emergence of memristors in VLSI technologies the idea of designing scalable analog data storage elements finds its second wind. A memristor, known for its history dependent resistance levels, independently can provide blocks of binary or discrete state data storage. However, using single memristor to save the analog value is practically limited due to the device variability and implementation complexity. In this paper, we present a new design of discrete state memory cell c…
Neuromorphic architectures offer great promise for achieving computation capacities beyond conventional Von Neumann machines. The essential elements for achieving this vision are highly scalable synaptic mimics that do not undermine biological fidelity. Here we demonstrate that single solid-state TiO2 memristors can exhibit non-associative plasticity phenomena observed in biological synapses, supported by their metastable memory state transition properties. We show that, contrary to conventional uses of solid-state memory, the existence of rate-limiting volatility is a key feature for capturin…
Nazarbayev University（纳扎尔巴耶夫大学）；King Abdullah University of Science and Technology（阿卜杜拉国王科技大学）
The on-chip implementation of learning algorithms would speed-up the training of neural networks in crossbar arrays. The circuit level design and implementation of backpropagation algorithm using gradient descent operation for neural network architectures is an open problem. In this paper, we proposed the analog backpropagation learning circuits for various memristive learning architectures, such as Deep Neural Network (DNN), Binary Neural Network (BNN), Multiple Neural Network (MNN), Hierarchical Temporal Memory (HTM) and Long-Short Term Memory (LSTM). The circuit design and verification is d…
Memristive materials are related to neuromorphic applications as they can combine information processing with memory storage in a single computational element, just as biological neurons. Many of these bioinspired materials emulate the characteristics of memory and learning processes that happen in the brain. In this work, we report the memristive properties of a two-terminal (2-T) organic device based on ionic migration mediated by an ion-transport polymer. The material possesses unique memristive properties: it is reversibly switchable, shows tens of conductive states, presents Hebbian learn…
University of Central Florida（中佛罗里达大学）；Sundrape Inc.（Sundrape公司）；Nanjing University（南京大学）
The advent of deep learning has resulted in a number of applications which have transformed the landscape of the research area in which it has been applied. However, with an increase in popularity, the complexity of classical deep neural networks has increased over the years. As a result, this has leads to considerable problems during deployment on devices with space and time constraints. In this work, we perform a review of the present advancements in non-volatile memory and how the use of resistive RAM memory, particularly memristors, can help to progress the state of research in deep learni…
At the Faraday Discussion, in the paper titled `Neuromorphic computation with spiking memristors: habituation, experimental instantiation of logic gates and a novel sequence-sensitive perceptron model' it was demonstrated that a large amount of computation could be done in a sequential way using memristor current spikes (d.c. response). As these spikes are found in many memristors (possibly all), this novel approach could be highly useful for fast and reproducible memristor circuits. However, questions were raised as to whether these spikes were actually due to memristance or merely capacitanc…
In-Memory Computing (IMC) hardware using Memristive Crossbar Arrays (MCAs) are gaining popularity to accelerate Deep Neural Networks (DNNs) since it alleviates the "memory wall" problem associated with von-Neumann architecture. The hardware efficiency (energy, latency and area) as well as application accuracy (considering device and circuit non-idealities) of DNNs mapped to such hardware are co-dependent on network parameters, such as kernel size, depth etc. and hardware architecture parameters such as crossbar size. However, co-optimization of both network and hardware parameters presents a c…
The automated wafer inspection and quality control is a complex and time-consuming task, which can speed up using neuromorphic memristive architectures, as a separate inspection device or integrating directly into sensors. This paper presents the performance analysis and comparison of different neuromorphic architectures for patterned wafer quality inspection and classification. The application of non-volatile memristive devices in these architectures ensures low power consumption, small on-chip area scalability. We demonstrate that Long-Short Term Memory (LSTM) outperforms other architectures…
Spiking recurrent neural networks (RNNs) are a promising tool for solving a wide variety of complex cognitive and motor tasks, due to their rich temporal dynamics and sparse processing. However training spiking RNNs on dedicated neuromorphic hardware is still an open challenge. This is due mainly to the lack of local, hardware-friendly learning mechanisms that can solve the temporal credit assignment problem and ensure stable network dynamics, even when the weight resolution is limited. These challenges are further accentuated, if one resorts to using memristive devices for in-memory computing…
The original Pascaline was a mechanical calculator able to sum and subtract integers. It encodes information in the angles of mechanical wheels and through a set of gears, and aided by gravity, could perform the calculations. Here, we show that such a concept can be realized in electronics using memory elements such as memristive systems. By using memristive emulators we have demonstrated experimentally the memcomputing version of the mechanical Pascaline, capable of processing and storing the numerical results in the multiple levels of each memristive element. Our result is the first experime…
Dynamic neural network with memristive CIM and CAM for 2D and 3D vision
the University of Hong Kong（香港大学）；ACCESS – AI Chip Center for Emerging Smart Systems, InnoHK Centers（ACCESS——新兴智能系统AI芯片中心（InnoHK中心））；Institute of the Mind, the University of Hong Kong（香港大学心智研究所）；Institute of Microelectronics, Chinese Academy of Sciences（中国科学院微电子研究所）；University of Chinese Academy of Sciences（中国科学院大学）
As conventional memory technologies are challenged by their technological physical limits, emerging technologies driven by novel materials are becoming an attractive option for future memory architectures. Among these technologies, Resistive Memories (ReRAM) created new possibilities because of their nano-features and unique $I$-$V$ characteristics. One particular problem that limits the maximum array size is interference from neighboring cells due to sneak-path currents. A possible device level solution to address this issue is to implement a memory array using complementary resistive switche…
The possibility of in-memory computing with volatile memristive devices, namely, memristors requiring a power source to sustain their memory, is demonstrated. We have adopted a hysteretic graphene-based field emission structure as a prototype of volatile memristor, which is characterized by a non-pinched hysteresis loop. Memristive model of the structure is developed and used to simulate a polymorphic circuit implementing in-memory computing gates such as the material implication. Specific regions of parameter space realizing useful logic functions are identified. Our results are applicable to…
Bielefeld University（比勒费尔德大学）；University of Zurich（苏黎世大学）；ETH Zurich（苏黎世联邦理工学院）
The development of memristive device technologies has reached a level of maturity to enable the design of complex and large-scale hybrid memristive-CMOS neural processing systems. These systems offer promising solutions for implementing novel in-memory computing architectures for machine learning and data analysis problems. We argue that they are also ideal building blocks for the integration in neuromorphic electronic circuits suitable for ultra-low power brain-inspired sensory processing systems, therefore leading to the innovative solutions for always-on edge-computing and Internet-of-Thing…
The interest in memristors has risen due to their possible application both as memory units and as computational devices in combination with CMOS. This is in part due to their nonlinear dynamics, and a strong dependence on the circuit topology. We provide evidence that also purely memristive circuits can be employed for computational purposes. In the present paper we show that a polynomial Lyapunov function in the memory parameters exists for the case of DC controlled memristors. Such Lyapunov function can be asymptotically approximated with binary variables, and mapped to quadratic combinator…
Physical unclonable functions (PUFs) exploit the intrinsic complexity and irreproducibility of physical systems to generate secret information. PUFs have the potential to provide fundamentally higher security than traditional cryptographic methods by preventing the cloning of identities and the extraction of secret keys. One unique and exciting opportunity is that of using the super-high information content (SHIC) capability of nanocrossbar architecture as well as the high resistance programming variation of resistive memories to develop a highly secure on-chip PUFs for extremely resource cons…
Memristors are novel non volatile devices that manage to combine storing and processing capabilities in the same physical place.Their nanoscale dimensions and low power consumption enable the further design of various nanoelectronic processing circuits and corresponding computing architectures, like neuromorhpic, in memory, unconventional, etc.One of the possible ways to exploit the memristor's advantages is by combining them with Cellular Automata (CA).CA constitute a well known non von Neumann computing architecture that is based on the local interconnection of simple identical cells forming…
Neuromorphic computing using post-CMOS technologies is gaining immense popularity due to its promising abilities to address the memory and power bottlenecks in von-Neumann computing systems. In this paper, we propose RESPARC - a reconfigurable and energy efficient architecture built-on Memristive Crossbar Arrays (MCA) for deep Spiking Neural Networks (SNNs). Prior works were primarily focused on device and circuit implementations of SNNs on crossbars. RESPARC advances this by proposing a complete system for SNN acceleration and its subsequent analysis. RESPARC utilizes the energy-efficiency of…
