Low bit-width Quantized Neural Networks (QNNs) enable deployment of complex machine learning models on constrained devices such as microcontrollers (MCUs) by reducing their memory footprint. Fine-grained asymmetric quantization (i.e., different bit-widths assigned to weights and activations on a tensor-by-tensor basis) is a particularly interesting scheme to maximize accuracy under a tight memory constraint. However, the lack of sub-byte instruction set architecture (ISA) support in SoA microprocessors makes it hard to fully exploit this extreme quantization paradigm in embedded MCUs. Support for sub-byte and asymmetric QNNs would require many precision formats and an exorbitant amount of opcode space. In this work, we attack this problem with status-based SIMD instructions: rather than encoding precision explicitly, each operand's precision is set dynamically in a core status register. We propose a novel RISC-V ISA core MPIC (Mixed Precision Inference Core) based on the open-source RI5CY core. Our approach enables full support for mixed-precision QNN inference with different combinations of operands at 16-, 8-, 4- and 2-bit precision, without adding any extra opcode or increasing the complexity of the decode stage. Our results show that MPIC improves both performance and energy efficiency by a factor of 1.1-4.9x when compared to software-based mixed-precision on RI5CY; with respect to commercially available Cortex-M4 and M7 microcontrollers, it delivers 3.6-11.7x better performance and 41-155x higher efficiency.
In this paper, we propose a high-performance RISC-V soft processor with an efficient fetch unit supporting the compressed instructions targeting on FPGA. The compressed instruction extension in RISC-V can reduce the program size by about 25%. But it needs a complicated logic for the instruction fetch unit and has a significant impact on performance. We propose an instruction fetch unit that supports the compressed instructions while exhibiting high performance. Furthermore, we propose a RISC-V soft processor using this unit. We implement this proposed processor in Verilog HDL and verify the be…
RISC-V, an open instruction set architecture, is getting the attention of soft processor developers. Implementing only a basic 32-bit integer instruction set of RISC-V, which is defined as RV32I, might be satisfactory for embedded systems. However, multiplication and division instructions are not present in RV32I, rather than defined as M-extension. Several research projects have proposed both RV32I and RV32IM processor. However, there is no indication of how much performance can be improved by adding M-extension to RV32I. In other words, when we should consider adding M-extension into the sof…
This paper describes the design of a 1024-core processor chip in 16nm FinFet technology. The chip ("Epiphany-V") contains an array of 1024 64-bit RISC processors, 64MB of on-chip SRAM, three 136-bit wide mesh Networks-On-Chip, and 1024 programmable IO pins. The chip has taped out and is being manufactured by TSMC. This research was developed with funding from the Defense Advanced Research Projects Agency (DARPA). The views, opinions and/or findings expressed are those of the author and should not be interpreted as representing the official views or policies of the Department of Defense or the …
We present a DNN accelerator that allows inference at arbitrary precision with dedicated processing elements that are configurable at the bit level. Our DNN accelerator has 8 Processing Elements controlled by a RISC-V controller with a combined 8.2 TMACs of computational power when implemented with the recent Alveo U250 FPGA platform. We develop a code generator tool that ingests CNN models in ONNX format and generates an executable command stream for the RISC-V controller. We demonstrate the scalable throughput of our accelerator by running different DNN kernels and models when different quan…
RISC-V is a RISC based open and loyalty free instruction set architecture which has been developed since 2010, and can be used for cost-effective soft processors on FPGAs. The basic 32-bit integer instruction set in RISC-V is defined as RV32I, which is sufficient to support the operating system environment and suits for embedded systems. In this paper, we propose an optimized RV32I soft processor named RVCoreP adopting five-stage pipelining. The processor applies three effective optimization methods to improve the operating frequency. These methods are instruction fetch unit optimization inclu…
In this paper, we present Quark, an integer RISC-V vector processor specifically tailored for sub-byte DNN inference. Quark is implemented in GlobalFoundries' 22FDX FD-SOI technology. It is designed on top of Ara, an open-source 64-bit RISC-V vector processor. To accommodate sub-byte DNN inference, Quark extends Ara by adding specialized vector instructions to perform sub-byte quantized operations. We also remove the floating-point unit from Quarks' lanes and use the CVA6 RISC-V scalar core for the re-scaling operations that are required in quantized neural network inference. This makes each l…
GRVI is an FPGA-efficient RISC-V RV32I soft processor. Phalanx is a parallel processor and accelerator array framework. Groups of processors and accelerators form shared memory clusters. Clusters are interconnected with each other and with extreme bandwidth I/O and memory devices by a 300- bit-wide Hoplite NOC. An example Kintex UltraScale KU040 system has 400 RISC-V cores, peak throughput of 100,000 MIPS, peak shared memory bandwidth of 600 GB/s, NOC bisection bandwidth of 700 Gbps, and uses 13 W.
Recent safety standards set stringent requirements for the target fault coverage in embedded microprocessors, with the objective to guarantee robustness and functional safety of the critical electronic systems. This motivates the need for improving the quality of test generation for microprocessors. A new high-level implementation-independent test generation method for RISC processors is proposed. The set of instructions of the processor is partitioned nto groups. For each group, a dedicated test template is created, to be used for generating two test programs, for testing the control and the …
Heterogeneous, multicore SoC architectures are a critical component of today's computing landscape. However, supporting both increasing heterogeneity and multicore execution are significant design challenges. Meanwhile, the growing RISC-V and open-source hardware (OSH) movements have resulted in an increased number of open-source RISC-V processor implementations; however, there are fewer open source SoC design platforms that integrate these processor cores. We present modifications to ESP, an open-source SoC design platform, to enable multicore execution with the RISC-V CVA6 processor. Our imp…
In this paper, we present Ara, a 64-bit vector processor based on the version 0.5 draft of RISC-V's vector extension, implemented in GlobalFoundries 22FDX FD-SOI technology. Ara's microarchitecture is scalable, as it is composed of a set of identical lanes, each containing part of the processor's vector register file and functional units. It achieves up to 97% FPU utilization when running a 256 x 256 double precision matrix multiplication on sixteen lanes. Ara runs at more than 1 GHz in the typical corner (TT/0.80V/25 oC) achieving a performance up to 33 DP-GFLOPS. In terms of energy efficienc…
RISC-V is an open and royalty free instruction set architecture which has been developed at the University of California, Berkeley. The processors using RISC-V can be designed and released freely. Because of this, various processor cores and system on chips (SoCs) have been released so far. However, there are a few public RISC-V computer systems that are portable and can boot Linux operating systems. In this paper, we describe a portable and Linux capable RISC-V computer system targeting FPGAs in Verilog HDL. This system can be implemented on an FPGA with fewer hardware resources, and can be i…
In this paper we present Arrow, a configurable hardware accelerator architecture that implements a subset of the RISC-V v0.9 vector ISA extension aimed at edge machine learning inference. Our experimental results show that an Arrow co-processor can execute a suite of vector and matrix benchmarks fundamental to machine learning inference 2 - 78x faster than a scalar RISC processor while consuming 20% - 99% less energy when implemented in a Xilinx XC7A200T-1SBG484C FPGA.
Research on LLM Acceleration Using the High-Performance RISC-V Processor "Xiangshan" (Nanhu Version) Based on the Open-Source Matrix Instruction Set Extension (Vector Dot Product)
RISC-V architectures have gained importance in the last years due to their flexibility and open-source Instruction Set Architecture (ISA), allowing developers to efficiently adopt RISC-V processors in several domains with a reduced cost. For application domains, such as safety-critical and mission-critical, the execution must be reliable as a fault can compromise the system's ability to operate correctly. However, the application's error rate on RISC-V processors is not significantly evaluated, as it has been done for standard x86 processors. In this work, we investigate the error rate of a co…
Internet-of-Things end-nodes demand low power processing platforms characterized by heterogeneous dedicated units, controlled by a processor core running concurrent control threads. Such architecture scheme fits one of the main target application domain of the RISC-V instruction set. We present an open-source processing core compliant with RISC-V on the software side and with the popular Pulpino processor platform on the hardware side, while supporting interleaved multi-threading for IoT applications. The latter feature is a novel contribution in this application domain. We report details abou…
Today's microprocessors have grown significantly in complexity and functionality. Most of today's processors provide at least three levels of memory hierarchy, are heavily pipelined, and support some sort of cache coherency protocol. These features are extremely complex and sophisticated, and present their own set of unique verification challenges. Verification is clearly not a point tool, but is part of a process that starts from initial product conception and is to some degrees complete when the product goes to market. Functional verification is necessary to verify the functionality at RTL l…
Lyra: A Hardware-Accelerated RISC-V Verification Framework with Generative Model-Based Processor Fuzzing
SKLP, Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）；University of Chinese Academy of Sciences（中国科学院大学）；Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究院）；Southern University of Science and Technology（南方科技大学）；Shenzhen University of Advanced Technology（深圳先进技术研究院）
Energy efficiency has become an increasingly important concern in computer architecture due to the end of Dennard scaling. Heterogeneity has been explored as a way to achieve better energy efficiency and heterogeneous microarchitecture chips have become common in the mobile setting. Recent research has explored using heterogeneous-ISA, heterogeneous microarchitecture, general-purpose cores to achieve further energy efficiency gains. However, there is no open-source hardware implementation of a heterogeneous-ISA processor available for research, and effective research on heterogeneous-ISA proce…
The Rocket Chip Generator uses a collection of parameterized processor components to produce RISC-V-based SoCs. It is a powerful tool that can produce a wide variety of processor designs ranging from tiny embedded processors to complex multi-core systems. In this paper we extend the features of the Memory Management Unit of the Rocket Chip Generator and specifically the TLB hierarchy. TLBs are essential in terms of performance because they mitigate the overhead of frequent Page Table Walks, but may harm the critical path of the processor due to their size and/or associativity. In the original …
Side-channel attacks on microprocessors, like the RISC-V, exhibit security vulnerabilities that lead to several design challenges. Hence, it is imperative to study and analyze these security vulnerabilities comprehensively. In this paper, we present a brief yet comprehensive study of the security vulnerabilities in modern microprocessors with respect to side-channel attacks and their respective mitigation techniques. The focus of this paper is to analyze the hardware-exploitable side-channel attack using power consumption and software-exploitable side-channel attacks to manipulate cache. Towar…
In this paper, we introduce the design and verification frameworks for developing a fully-functional emerging ternary processor. Based on the existing compiling environments for binary processors, for the given ternary instructions, the software-level framework provides an efficient way to convert the given programs to the ternary assembly codes. We also present a hardware-level framework to rapidly evaluate the performance of a ternary processor implemented in arbitrary design technology. As a case study, the fully-functional 9-trit advanced RISC-based ternary (ART-9) core is newly developed …
