We present dimensional circuit synthesis, a new method for generating digital logic circuits that improve the efficiency of training and inference of machine learning models from sensor data. The hardware accelerators that the method generates are compact enough (a few thousand gates) to allow integration within low-cost miniaturized sensor integrated circuits, right next to the sensor transducer. The method takes as input a description of physical properties of relevant signals in the sensor transduction process and generates as output a Verilog register transfer level (RTL) description for a circuit that computes low-level features that exploit the units of measure of the signals in the system.
We implement dimensional circuit synthesis as a backend to the compiler for Newton, a language for describing physical systems. We evaluate the backend implementation and the hardware it generates, on descriptions of 7 physical systems. The results show that our implementation of dimensional circuit synthesis generates circuits of as little as 1662 logic cells / 1239 gates for the systems we evaluate.
We synthesize the designs generated by the dimensional circuit synthesis compilation backend for a low-power miniature FPGA targeted by its manufacturer at sensor interface applications. The circuits which the method generated use as little as 27% of the resources of the 2.15x2.5 mm FPGA. We measure the power dissipation of the FPGA's isolated core supply rail and show that, driven with a pseudorandom signal input stream, the synthesized designs use as little as 1.0 mW and no more than 5.8 mW. These results show the feasibility of integrating physics-inspired machine learning methods within low-cost miniaturized sensor integrated circuits, right next to the sensor transducer.
This paper presents a structural design of the hardware-efficient module for implementation of convolution neural network (CNN) basic operation with reduced implementation complexity. For this purpose we utilize some modification of the Winograd minimal filtering method as well as computation vectorization principles. This module calculate inner products of two consecutive segments of the original data sequence, formed by a sliding window of length 3, with the elements of a filter impulse response. The fully parallel structure of the module for calculating these two inner products, based on th…
Path planning for autonomous driving with dynamic obstacles poses a challenge because it needs to perform a higher-dimensional search (with time-dimension) while still meeting real-time constraints. This paper proposes an algorithm-hardware co-optimization approach to accelerate path planning with high-dimensional search space. First, we reduce the time for a nearest neighbor search and collision detection by mapping nodes and obstacles to a lower-dimensional space and memoizing recent search results. Then, we propose a hardware extension for efficient memoization. The experimental results on …
In today's era of "scale-out", this paper makes the case that a specialized hardware architecture based on "scale-in"--placing as many specialized processors as possible along with their memory systems and interconnect links within one or two boards in a rack--would offer the potential to boost large recommender system throughput by 12-62x for inference and 12-45x for training compared to the DGX-2 state-of-the-art AI platform, while minimizing the performance impact of distributing large models across multiple processors. By analyzing Facebook's representative model--Deep Learning Recommendat…
Graph Neural Networks (GNNs) use a fully-connected layer to extract features from the nodes of a graph and aggregate these features using message passing between nodes, combining two distinct computational patterns: dense, regular computations and sparse, irregular computations. To address this challenge, we propose GNNerator, an accelerator with heterogeneous compute engines optimized for these two patterns. Further, GNNerator implements feature-blocking, a novel GNN dataflow that beneficially trades off irregular memory accesses during aggregation for regular memory accesses during feature e…
Sparsity is an intrinsic property of convolutional neural network(CNN) and worth exploiting for CNN accelerators, but extra processing comes with hardware overhead, causing many architectures suffering from only minor profit. Meanwhile, systolic array has been increasingly competitive on CNNs acceleration for its high spatiotemporal locality and low hardware overhead. However, the irregularity of sparsity induces imbalanced workload under the rigid systolic dataflow, causing performance degradation. Thus, this paper proposed a systolicarray-based architecture, called Sense, for sparse CNN acce…
In this paper, we propose a methodology for partitioning and mapping computational intensive applications in reconfigurable hardware blocks of different granularity. A generic hybrid reconfigurable architecture is considered so as the methodology can be applicable to a large number of heterogeneous reconfigurable platforms. The methodology mainly consists of two stages, the analysis and the mapping of the application onto fine and coarse-grain hardware resources. A prototype framework consisting of analysis, partitioning and mapping tools has been also developed. For the coarse-grain reconfigu…
Genome analysis fundamentally starts with a process known as read mapping, where sequenced fragments of an organism's genome are compared against a reference genome. Read mapping is currently a major bottleneck in the entire genome analysis pipeline, because state-of-the-art genome sequencing technologies are able to sequence a genome much faster than the computational techniques employed to analyze the genome. We describe the ongoing journey in significantly improving the performance of read mapping. We explain state-of-the-art algorithmic methods and hardware-based acceleration approaches. A…
Bit-serial architectures can handle Neural Networks (NNs) with different weight precisions, achieving higher resource efficiency compared with bit-parallel architectures. Besides, the weights contain abundant zero bits owing to the fault tolerance of NNs, indicating that bit sparsity of NNs can be further exploited for performance improvement. However, the irregular proportion of zero bits in each weight causes imbalanced workloads in the Processing Element (PE) array, which degrades performance or induces overhead for sparse processing. Thus, this paper proposed a bit-sparsity quantization me…
The Transformer has been an indispensable staple in deep learning. However, for real-life applications, it is very challenging to deploy efficient Transformers due to immense parameters and operations of models. To relieve this burden, exploiting sparsity is an effective approach to accelerate Transformers. Newly emerging Ampere GPUs leverage a 2:4 sparsity pattern to achieve model acceleration, while it can hardly meet the diverse algorithm and hardware constraints when deploying models. By contrast, we propose an algorithm-hardware co-optimized framework to flexibly and efficiently accelerat…
Genome sequence analysis plays a pivotal role in enabling many medical and scientific advancements in personalized medicine, outbreak tracing, and forensics. However, the analysis of genome sequencing data is currently bottlenecked by the computational power and memory bandwidth limitations of existing systems. In this dissertation, we propose four major works, where we characterize the real-system behavior of the genome sequence analysis pipeline and its associated tools, expose the bottlenecks and tradeoffs, and co-design fast and efficient algorithms along with scalable and energy-efficient…
Politecnico di Torino（都灵理工大学）；New York University Abu Dhabi（纽约大学阿布扎比分校）
Currently, Machine Learning (ML) is becoming ubiquitous in everyday life. Deep Learning (DL) is already present in many applications ranging from computer vision for medicine to autonomous driving of modern cars as well as other sectors in security, healthcare, and finance. However, to achieve impressive performance, these algorithms employ very deep networks, requiring a significant computational power, both during the training and inference time. A single inference of a DL model may require billions of multiply-and-accumulated operations, making the DL extremely compute- and energy-hungry. I…
Spiking Neural Networks (SNN) are an emerging computation model, which uses event-driven activation and bio-inspired learning algorithms. SNN-based machine-learning programs are typically executed on tile- based neuromorphic hardware platforms, where each tile consists of a computation unit called crossbar, which maps neurons and synapses of the program. However, synthesizing such programs on an off-the-shelf neuromorphic hardware is challenging. This is because of the inherent resource and latency limitations of the hardware, which impact both model performance, e.g., accuracy, and hardware p…
We present Syntroids, a case study for the automatic synthesis of hardware from a temporal logic specification. Syntroids is a space shooter arcade game realized on an FPGA, where the control flow architecture has been completely specified in Temporal Stream Logic (TSL) and implemented using reactive synthesis. TSL is a recently introduced temporal logic that separates control and data. This leads to scalable synthesis, because the cost of the synthesis process is independent of the complexity of the handled data. In this case study, we report on our experience with the TSL-based development o…
Convolutional Neural Networks (CNNs) are currently adopted to solve an ever greater number of problems, ranging from speech recognition to image classification and segmentation. The large amount of processing required by CNNs calls for dedicated and tailored hardware support methods. Moreover, CNN workloads have a streaming nature, well suited to reconfigurable hardware architectures such as FPGAs. The amount and diversity of research on the subject of CNN FPGA acceleration within the last 3 years demonstrates the tremendous industrial and academic interest. This paper presents a state-of-the-…
Graph Neural Network (GNN) inference is used in many real-world applications. Data sparsity in GNN inference, including sparsity in the input graph and the GNN model, offer opportunities to further speed up inference. Also, many pruning techniques have been proposed for model compression that increase the data sparsity of GNNs. We propose Dynasparse, a comprehensive hardware-software codesign on FPGA to accelerate GNN inference through dynamic sparsity exploitation. For this, we decouple the GNN computation kernels from the basic computation primitives, and explore hardware-software codesign a…
Compiler frameworks are crucial for the widespread use of FPGA-based deep learning accelerators. They allow researchers and developers, who are not familiar with hardware engineering, to harness the performance attained by domain-specific logic. There exists a variety of frameworks for conventional artificial neural networks. However, not much research effort has been put into the creation of frameworks optimized for spiking neural networks (SNNs). This new generation of neural networks becomes increasingly interesting for the deployment of AI on edge devices, which have tight power and resour…
Motivation: High throughput DNA sequencing (HTS) technologies generate an excessive number of small DNA segments -- called short reads -- that cause significant computational burden. To analyze the entire genome, each of the billions of short reads must be mapped to a reference genome based on the similarity between a read and "candidate" locations in that reference genome. The similarity measurement, called alignment, formulated as an approximate string matching problem, is the computational bottleneck because: (1) it is implemented using quadratic-time dynamic programming algorithms, and (2)…
A Survey on Design Methodologies for Accelerating Deep Learning on Heterogeneous Architectures
Politecnico di Milano（米兰理工大学）；Università di Bologna（博洛尼亚大学）；Università degli Studi di Ferrara（费拉拉大学）；INFN（意大利国家核物理研究所）；Università degli Studi di Catania（卡塔尼亚大学）；Università degli Studi di Roma Tor Vergata（罗马-托尔维加塔大学）；Università degli Studi della Calabria（卡拉布里亚大学）
Pipelined algorithms implemented in field programmable gate arrays are being extensively used for hardware triggers in the modern experimental high energy physics field and the complexity of such algorithms are increases rapidly. For development of such hardware triggers, algorithms are developed in $\texttt{C++}$, ported to hardware description language for synthesizing firmware, and then ported back to $\texttt{C++}$ for simulating the firmware response down to the single bit level. We present a $\texttt{C++}$ software framework which automatically simulates and generates hardware descriptio…
Transformer is a deep learning language model widely used for natural language processing (NLP) services in datacenters. Among transformer models, Generative Pre-trained Transformer (GPT) has achieved remarkable performance in text generation, or natural language generation (NLG), which needs the processing of a large input context in the summarization stage, followed by the generation stage that produces a single word at a time. The conventional platforms such as GPU are specialized for the parallel processing of large inputs in the summarization stage, but their performance significantly deg…
GFormer: Accelerating Large Language Models with Optimized Transformers on Gaudi Processors
University of Houston（休斯顿大学）；Indiana University（印第安纳大学）；Stevens Institute of Technology（史蒂文斯理工学院）；Argonne National Laboratory（阿贡国家实验室）；Binghamton University（宾汉姆顿大学）
