With the rise of self-driving vehicles comes the risk of accidents and the need for higher safety, and protection for pedestrian detection in the following scenarios: imminent crashes, thus the car should crash into an object and avoid the pedestrian, and in the case of road intersections, where it is important for the car to stop when pedestrians are crossing. Currently, a special topology of deep neural networks called Fused Deep Neural Network (F-DNN) is considered to be the state of the art in pedestrian detection, as it has the lowest miss rate, yet it is very slow. Therefore, acceleration is needed to speed up the performance. This project proposes two contributions to address this problem, by using a deep neural network used for object detection, called Single Shot Multi-Box Detector (SSD). The first contribution is training and tuning the hyperparameters of SSD to improve pedestrian detection. The second contribution is a new FPGA design for accelerating the model on the Altera Arria 10 platform. The final system will be used in self-driving vehicles in real-time. Preliminary results of the improved SSD shows 3% higher miss-rate than F-DNN on Caltech pedestrian detection benchmark, but 4x performance improvement. The acceleration design is expected to achieve an additional performance improvement significantly outweighing the minimal difference in accuracy.
Pedestrian detection is one of the key problems in emerging self-driving car industry. And HOG algorithm has proven to provide good accuracy for pedestrian detection. There are plenty of research works have been done in accelerating HOG algorithm on FPGA because of its low-power and high-throughput characteristics. In this paper, we present a high-performance HOG architecture for pedestrian detection on a low-cost FPGA platform. It achieves a maximum throughput of 526 FPS with 640x480 input images, which is 3.25 times faster than the state of the art design. The accelerator is integrated with …
James Cook University（詹姆斯库克大学）；College of Science and Engineering（科学与工程学院）
Recent technological advances have proliferated the available computing power, memory, and speed of modern Central Processing Units (CPUs), Graphics Processing Units (GPUs), and Field Programmable Gate Arrays (FPGAs). Consequently, the performance and complexity of Artificial Neural Networks (ANNs) is burgeoning. While GPU accelerated Deep Neural Networks (DNNs) currently offer state-of-the-art performance, they consume large amounts of power. Training such networks on CPUs is inefficient, as data throughput and parallel computation is limited. FPGAs are considered a suitable candidate for per…
In this project, we have successfully designed, implemented, deployed and tested a novel FPGA accelerated algorithm for neural network training. The algorithm itself was developed in an independent study option. This training method is based on Alternating Direction Method of Multipliers algorithm, which has strong parallel characteristics and avoids procedures such as matrix inversion that are problematic in hardware designs by employing LSMR. As an intermediate stage, we fully implemented the ADMM-LSMR method in C language for feed-forward neural networks with a flexible number of layers and…
Image Understanding is becoming a vital feature in ever more applications ranging from medical diagnostics to autonomous vehicles. Many applications demand for embedded solutions that integrate into existing systems with tight real-time and power constraints. Convolutional Neural Networks (CNNs) presently achieve record-breaking accuracies in all image understanding benchmarks, but have a very high computational complexity. Embedded CNNs thus call for small and efficient, yet very powerful computing platforms. This master thesis explores the potential of FPGA-based CNN acceleration and demonst…
University of California San Diego（加州大学圣地亚哥分校）；Princeton University（普林斯顿大学）；University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；Fermi National Accelerator Laboratory（费米国家加速器实验室）；Massachusetts Institute of Technology（麻省理工学院）；Institute of Physics Belgrade（贝尔格莱德物理研究所）；European Organization for Nuclear Research (CERN)（欧洲核子研究组织（CERN））；California Institute of Technology（加州理工学院）；Purdue University（普渡大学）；HawkEye360；University of Illinois at Chicago（伊利诺伊大学芝加哥分校）
We develop and study FPGA implementations of algorithms for charged particle tracking based on graph neural networks. The two complementary FPGA designs are based on OpenCL, a framework for writing programs that execute across heterogeneous platforms, and hls4ml, a high-level-synthesis-based compiler for neural network to firmware conversion. We evaluate and compare the resource usage, latency, and tracking performance of our implementations based on a benchmark dataset. We find a considerable speedup over CPU-based execution is possible, potentially enabling such algorithms to be used effecti…
Closed-loop sleep modulation is an emerging research paradigm to treat sleep disorders and enhance sleep benefits. However, two major barriers hinder the widespread application of this research paradigm. First, subjects often need to be wire-connected to rack-mount instrumentation for data acquisition, which negatively affects sleep quality. Second, conventional real-time sleep stage classification algorithms give limited performance. In this work, we conquer these two limitations by developing a sleep modulation system that supports closed-loop operations on the device. Sleep stage classifica…
Compressing Recurrent Neural Networks for FPGA-accelerated Implementation in Fluorescence Lifetime Imaging
Center for Modeling, Simulation, & Imaging in Medicine（医学建模、仿真与成像中心）；Rensselaer Polytechnic Institute（伦斯勒理工学院）；IBM T.J. Watson Research Center（IBM TJ·沃森研究中心）
Unlocking Real-Time Fluorescence Lifetime Imaging: Multi-Pixel Parallelism for FPGA-Accelerated Processing
Center for Modeling, Simulation, & Imaging in Medicine（医学建模、仿真与成像中心）；Rensselaer Polytechnic Institute（伦斯勒理工学院）；IBM T.J. Watson Research Center（IBM T.J. 沃森研究中心）
Hardware-Accelerated Event-Graph Neural Networks for Low-Latency Time-Series Classification on SoC FPGA
Graduate School of Science and Technology（科学与技术研究生院）；Embedded Vision Systems Group, Computer Vision Laboratory（嵌入式视觉系统组，计算机视觉实验室）；AGH University of Krakow（克拉科夫AGH大学）；CEA-List, Université Grenoble Alpes（CEA-List，格勒诺布尔阿尔卑斯大学）
Real-Time Cell Sorting with Scalable In Situ FPGA-Accelerated Deep Learning
Department of Mechanical Engineering, Lehigh University（机械工程系，莱维大学）；Department of Electrical and Computer Engineering, Northwestern University（电气与计算机工程系，西北大学）；Coriell Institute for Medical Research（Coriell医学研究所）；Department of Computer Science, Lehigh University（计算机科学系，莱维大学）；Cooper Medical School of Rowan University（罗文大学合作医学院）；Center for Metabolic Disease Research, Temple University（代谢疾病研究中心， Temple大学）；Department of Mechanical Engineering and Mechanics, Drexel University（机械工程与力学系，德雷塞尔大学）；Real-time Processing Systems Division, Fermi National Accelerator Laboratory（费米国家加速器实验室实时处理系统部）；Department of Bioengineering, Lehigh University（生物工程系，莱维大学）
Recent advances in adversarial Deep Learning (DL) have opened up a largely unexplored surface for malicious attacks jeopardizing the integrity of autonomous DL systems. With the wide-spread usage of DL in critical and time-sensitive applications, including unmanned vehicles, drones, and video surveillance systems, online detection of malicious inputs is of utmost importance. We propose DeepFense, the first end-to-end automated framework that simultaneously enables efficient and safe execution of DL models. DeepFense formalizes the goal of thwarting adversarial attacks as an optimization proble…
Over the past few years machine learning has seen a renewed explosion of interest, following a number of studies showing the effectiveness of neural networks in a range of tasks which had previously been considered incredibly hard. Neural networks' effectiveness in the fields of image recognition and natural language processing stems primarily from the vast amounts of data available to companies and researchers, coupled with the huge amounts of compute power available in modern accelerators such as GPUs, FPGAs and ASICs. There are a number of approaches available to developers for utilizing GP…
Fully convolutional neural networks give accurate, per-pixel prediction for input images and have applications like semantic segmentation. However, a typical FCN usually requires lots of floating point computation and large run-time memory, which effectively limits its usability. We propose a method to train Bit Fully Convolution Network (BFCN), a fully convolutional neural network that has low bit-width weights and activations. Because most of its computation-intensive convolutions are accomplished between low bit-width numbers, a BFCN can be accelerated by an efficient bit-convolution implem…
University of Science and Technology of China（中国科学技术大学）
Sampling is an important process in many GNN structures in order to train larger datasets with a smaller computational complexity. However, compared to other processes in GNN (such as aggregate, backward propagation), the sampling process still costs tremendous time, which limits the speed of training. To reduce the time of sampling, hardware acceleration is an ideal choice. However, state of the art GNN acceleration proposal did not specify how to accelerate the sampling process. What's more, directly accelerating traditional sampling algorithms will make the structure of the accelerator very…
