In recent years, transformer-based models have shown state-of-the-art results for Natural Language Processing (NLP). In particular, the introduction of the BERT language model brought with it breakthroughs in tasks such as question answering and natural language inference, advancing applications that allow humans to interact naturally with embedded devices. FPGA-based overlay processors have been shown as effective solutions for edge image and video processing applications, which mostly rely on low precision linear matrix operations. In contrast, transformer-based NLP techniques employ a variety of higher precision nonlinear operations with significantly higher frequency. We present NPE, an FPGA-based overlay processor that can efficiently execute a variety of NLP models. NPE offers software-like programmability to the end user and, unlike FPGA designs that implement specialized accelerators for each nonlinear function, can be upgraded for future NLP models without requiring reconfiguration. We demonstrate that NPE can meet real-time conversational AI latency targets for the BERT language model with $4\times$ lower power than CPUs and $6\times$ lower power than GPUs. We also show NPE uses $3\times$ fewer FPGA resources relative to comparable BERT network-specific accelerators in the literature. NPE provides a cost-effective and power-efficient FPGA-based solution for Natural Language Processing at the edge.
Resource-Aware Just-in-Time OpenCL Compiler for Coarse-Grained FPGA Overlays
FPGA供应商最近开始关注OpenCL for FPGA，因为它能够利用异构计算平台固有的并行性。OpenCL允许在主机上运行的程序启动加速器内核，这些加速器内核可以在运行时针对特定架构进行编译，从而实现可移植性。然而，在使用FPGA供应商提供的OpenCL工具时，过高的编译时间（特别是FPGA放置和布线时间）是一个主要的绊脚石。长的编译时间意味着这些工具不能有效地使用即时（JIT）编译或运行时性能扩展。粗粒度覆盖代表了一种可能的解决方案，因为它们具有粗粒度和快速编译的优点。在本文中，我们提出了一种运行时编译的OpenCL内核的DSP块为基础的粗粒度覆盖，而不是直接到细粒度的FPGA结构的方法。所提出的方法允许JIT编译和按需资源感知的内核复制，以更好地利用可用的覆盖资源，提高抽象级别，同时显着减少编译时间。我们进一步证明，这种方法甚至可以用于运行时编译OpenCL内核的嵌入式异构Zynq设备的ARM处理器上。
SkipOPU: An FPGA-based Overlay Processor for Large Language Models with Dynamically Allocated Computation
University of California, Los Angeles（加州大学洛杉矶分校）；Eastern Institute of Technology（宁波诺丁汉大学）；Institute of Digital Twin, Eastern Institute of Technology（宁波诺丁汉大学数字孪生研究院）；Ningbo Key Laboratory of Spatial Intelligence and Digital Derivative, Institute of Digital Twin, Eastern Institute of Technology（宁波诺丁汉大学数字孪生研究院空间智能与数字衍生宁波市重点实验室）；Chiplet CAD and Manufacturing Engineering Research Center of Zhejiang Province, Ningbo Institute of Digital Twin, Eastern Institute of Technology（浙江省芯片级封装CAD与制造工程研究中心，宁波诺丁汉大学数字孪生研究院）
