首页/论文/2007.07759

arXiv 2007.077592020-07-15cs.AReess.IV

Enabling Mixed-Precision Quantized Neural Networks in Extreme-Edge Devices

Nazareno Bruschi, Angelo Garofalo, Francesco Conti, Giuseppe Tagliavini, Davide Rossi

备注：: 4 pages, 6 figures, published in 17th ACM International Conference on Computing Frontiers (CF '20), May 11--13, 2020, Catania, Italy
DOI：: 10.1145/3387902.3394038

英文摘要：

The deployment of Quantized Neural Networks (QNN) on advanced microcontrollers requires optimized software to exploit digital signal processing (DSP) extensions of modern instruction set architectures (ISA). As such, recent research proposed optimized libraries for QNNs (from 8-bit to 2-bit) such as CMSIS-NN and PULP-NN. This work presents an extension to the PULP-NN library targeting the acceleration of mixed-precision Deep Neural Networks, an emerging paradigm able to significantly shrink the memory footprint of deep neural networks with negligible accuracy loss. The library, composed of 27 kernels, one for each permutation of input feature maps, weights, and output feature maps precision (considering 8-bit, 4-bit and 2-bit), enables efficient inference of QNN on parallel ultra-low-power (PULP) clusters of RISC-V based processors, featuring the RV32IMCXpulpV2 ISA. The proposed solution, benchmarked on an 8-cores GAP-8 PULP cluster, reaches peak performance of 16 MACs/cycle on 8 cores, performing 21x to 25x faster than an STM32H7 (powered by an ARM Cortex M7 processor) with 15x to 21x better energy efficiency.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

阅读原文

阅读 PDF ↗

arXiv 页面 ↗HTML 原文 ↗

本文内容

英文摘要相关论文

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

继续探索

相关论文

高效的VQ-QAT和混合向量/线性量化神经网络

Efficient VQ-QAT and Mixed Vector/Linear quantized Neural Networks

Tianrun Gou, Puneet Gupta, Fellow, IEEE（田-run 郭, Puneet Gupta, IEEE会士）

本文提出三种向量量化模型权重压缩技术，通过余弦相似度分配和Top-1采样优化，减少代码书坍塌并支持端到端训练，同时利用可微神经架构搜索自适应选择量化配置，提升压缩效果。

2026-04-25cs.ARcs.LG

在商用现货可重构 MPSoC 上使用混合精度量化神经网络进行实时航天器位姿估计

Real-Time Spacecraft Pose Estimation Using Mixed-Precision Quantized Neural Network on COTS Reconfigurable MPSoC

École Polytechnique de Montréal（蒙特利尔理工学院）

该研究提出面向航天器位姿估计的混合精度量化神经网络 FPGA 协同设计，通过逐层量化敏感性评估和 FINN 数据流加速器实现 7.7 倍提速与 19.5 倍能效提升，并开源实时实现。

2024-06-06cs.AIcs.ARcs.CV

一个用于细粒度混合精度量化神经网络推理的 3 TOPS/W RISC-V 并行集群

A 3 TOPS/W RISC-V Parallel Cluster for Inference of Fine-Grain Mixed-Precision Quantized Neural Networks

本文提出了一种基于 RISC-V 的软硬件栈 Flex-V 及其八核紧耦合集群，通过融合混合精度点积指令与 CSR 编码实现高效的 QNN 推理，在 22nm 工艺下达到 3.26 TOPS/W 能效，并显著提升了真实任务的执行速度。

2023-07-03cs.AR

TileFuse：用于AMD NPU上高效量化LLM推理的融合混合精度内核库

TileFuse: A Fused Mixed-Precision Kernel Library for Efficient Quantized LLM Inference on AMD NPUs

University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；Department of Electrical and Computer Engineering（电气与计算机工程系）

针对边缘NPU上量化LLM部署困难，提出TileFuse库，通过融合解包、反量化与GEMM/GEMV内核，并设计交错预分块布局与数据流，在XDNA2上实现AWQ格式原生支持，性能提升最高281%，能耗降低64.6%。

2026-07-03cs.AIcs.ARcs.DC

面向硬件加速器的运行时可重构多精度量化乘法的位级脉动阵列架构

Bitwise Systolic Array Architecture for Runtime-Reconfigurable Multi-precision Quantized Multiplication on Hardware Accelerators

Dresden University of Technology, Germany（德累斯顿理工大学）；Ruhr University Bochum, Germany（博德大学）；Center for Scalable Data Analytics and Artificial Intelligence (ScaDS.AI Dresden/Leipzig), Germany（可扩展数据与人工智能研究中心（ScaDS.AI 德累斯顿/莱比锡））

提出一种运行时可重构的多精度多通道位级脉动阵列，支持混合精度量化神经网络推理，在FPGA上实现1.32-3.57倍加速，最高时钟频率250MHz。

2026-06-24cs.AIcs.AR

A Mixed-Precision RISC-V Processor for Extreme-Edge DNN Inference

Low bit-width Quantized Neural Networks (QNNs) enable deployment of complex machine learning models on constrained devices such as microcontrollers (MCUs) by reducing their memory footprint. Fine-grained asymmetric quantization (i.e., different bit-widths assigned to weights and activations on a tensor-by-tensor basis) is a particularly interesting scheme to maximize accuracy under a tight memory constraint. However, the lack of sub-byte instruction set architecture (ISA) support in SoA microprocessors makes it hard to fully exploit this extreme quantization paradigm in embedded MCUs. Support …

2020-10-08cs.AR

更多相关论文收起更多论文

BrainTTA: A 35 fJ/op Compiler Programmable Mixed-Precision Transport-Triggered NN SoC

Recently, accelerators for extremely quantized deep neural network (DNN) inference with operand widths as low as 1-bit have gained popularity due to their ability to largely cut down energy cost per inference. In this paper, a flexible SoC with mixed-precision support is presented. Contrary to the current trend of fixed-datapath accelerators, this architecture makes use of a flexible datapath based on a Transport-Triggered Architecture (TTA). The architecture is fully programmable using C. The accelerator has a peak energy efficiency of 35/67/405 fJ/op (binary, ternary, and 8-bit precision) an…

2022-11-21cs.AR

片上硬件感知量化用于混合精度神经网络

On-Chip Hardware-Aware Quantization for Mixed Precision Neural Networks

The University of Hong Kong（香港大学）；ETH Zürich（苏黎世联邦理工学院）；Beihang University（北京航空航天大学）

本文提出片上硬件感知量化框架OHQ，在边缘设备上直接进行混合精度量化，通过片上感知流水线和掩码引导估计技术，实现精度与效率的优化，在ResNet-18和MobileNetV3上分别达到70%和73%精度，延迟降低15%-30%。

2024-05-23cs.AIcs.ARcs.LG

Low- and Mixed-Precision Inference Accelerators

With the surging popularity of edge computing, the need to efficiently perform neural network inference on battery-constrained IoT devices has greatly increased. While algorithmic developments enable neural networks to solve increasingly more complex tasks, the deployment of these networks on edge devices can be problematic due to the stringent energy, latency, and memory requirements. One way to alleviate these requirements is by heavily quantizing the neural network, i.e. lowering the precision of the operands. By taking quantization to the extreme, e.g. by using binary values, new opportuni…

2022-06-24cs.AR

LogicNets: Co-Designed Neural Networks and Circuits for Extreme-Throughput Applications

Xilinx Research Labs（赛灵思研究实验室）

Deployment of deep neural networks for applications that require very high throughput or extremely low latency is a severe computational challenge, further exacerbated by inefficiencies in mapping the computation to hardware. We present a novel method for designing neural network topologies that directly map to a highly efficient FPGA implementation. By exploiting the equivalence of artificial neurons with quantized inputs/outputs and truth tables, we can train quantized neural networks that can be directly converted to a netlist of truth tables, and subsequently deployed as a highly pipelinab…

2020-04-06cs.ARcs.LGeess.SP

Tiny but Accurate: A Pruned, Quantized and Optimized Memristor Crossbar Framework for Ultra Efficient DNN Implementation

Northeastern University（东北大学）；University of Connecticut（康涅狄格大学）；George Mason University（乔治梅森大学）；Florida International University（佛罗里达国际大学）

The state-of-art DNN structures involve intensive computation and high memory storage. To mitigate the challenges, the memristor crossbar array has emerged as an intrinsically suitable matrix computation and low-power acceleration framework for DNN applications. However, the high accuracy solution for extreme model compression on memristor crossbar array architecture is still waiting for unraveling. In this paper, we propose a memristor-based DNN framework which combines both structured weight pruning and quantization by incorporating alternating direction method of multipliers (ADMM) algorith…

2019-08-27cs.ARcs.ETcs.LG

Design of High-Throughput Mixed-Precision CNN Accelerators on FPGA

RWTH Aachen University（亚琛工业大学）

Convolutional Neural Networks (CNNs) reach high accuracies in various application domains, but require large amounts of computation and incur costly data movements. One method to decrease these costs while trading accuracy is weight and/or activation word-length reduction. Thereby, layer-wise mixed-precision quantization allows for more efficient results while inflating the design space. In this work, we present an in-depth quantitative methodology to efficiently explore the design space considering the limited hardware resources of a given FPGA. Our holistic exploration approach vertically tr…

2022-08-09cs.AIcs.ARcs.LG