Recently, significant accuracy improvement has been achieved for acoustic recognition systems by increasing the model size of Long Short-Term Memory (LSTM) networks. Unfortunately, the ever-increasing size of LSTM model leads to inefficient designs on FPGAs due to the limited on-chip resources. The previous work proposes to use a pruning based compression technique to reduce the model size and thus speedups the inference on FPGAs. However, the random nature of the pruning technique transforms the dense matrices of the model to highly unstructured sparse ones, which leads to unbalanced computation and irregular memory accesses and thus hurts the overall performance and energy efficiency.
In contrast, we propose to use a structured compression technique which could not only reduce the LSTM model size but also eliminate the irregularities of computation and memory accesses. This approach employs block-circulant instead of sparse matrices to compress weight matrices and reduces the storage requirement from $\mathcal{O}(k^2)$ to $\mathcal{O}(k)$. Fast Fourier Transform algorithm is utilized to further accelerate the inference by reducing the computational complexity from $\mathcal{O}(k^2)$ to $\mathcal{O}(k\text{log}k)$. The datapath and activation functions are quantized as 16-bit to improve the resource utilization. More importantly, we propose a comprehensive framework called C-LSTM to automatically optimize and implement a wide range of LSTM variants on FPGAs. According to the experimental results, C-LSTM achieves up to 18.8X and 33.5X gains for performance and energy efficiency compared with the state-of-the-art LSTM implementation under the same experimental setup, and the accuracy degradation is very small.
Hardware-Aware Deployment of Joint SAR Compression and Despeckling on FPGA
Technical University of Munich(慕尼黑工业大学);Remote Sensing Technology Institute, German Aerospace Center (DLR)(德国航空航天中心遥感技术研究所);University of the Bundeswehr Munich(慕尼黑联邦国防军大学)
E-RNN: Design Optimization for Efficient Recurrent Neural Networks in FPGAs
Syracuse University(雪城大学);Northeastern University(东北大学);Florida International University(佛罗里达国际大学);University of Southern California(南加利福尼亚大学);Carnegie Mellon University(卡内基梅隆大学);SUNY University at Buffalo(纽约州立大学布法罗分校)
FTRANS: Energy-Efficient Acceleration of Transformers using FPGA
University of Connecticut(康涅狄格大学);Stevens Institute of Technology(斯蒂文斯理工学院);Syracuse University(雪城大学);Microsoft Corporation(微软公司);Oak Ridge National Laboratory(橡树岭国家实验室);University of Texas at San Antonio(德克萨斯大学圣安东尼奥分校)
A Model Compression Method with Matrix Product Operators for Speech Enhancement
Institute of Acoustics, Chinese Academy of Sciences(中国科学院声学研究所);University of Chinese Academy of Sciences(中国科学院大学);School of Science, Renmin University of China(中国人民大学理学院);Xinjiang Technical Institute of Physics and Chemistry, Chinese Academy of Sciences(中国科学院新疆理化技术研究所)
Tensorized Embedding Layers for Efficient Model Compression
Skolkovo Institute of Science and Technology(斯科尔科沃科学技术研究院);Moscow Institute of Physics and Technology(莫斯科物理技术学院);Institute of Numerical Mathematics, RAS, Moscow, Russia(俄罗斯科学院数学数值研究所)
Efficient Split Learning LSTM Models for FPGA-based Edge IoT Devices
The Abdus Salam International Centre for Theoretical Physics(阿卜杜勒·萨拉姆国际理论物理中心);Faculty of Technical Sciences, University of Novi Sad(诺威萨德大学技术科学学院);The Institute for Artificial Intelligence Research and Development of Serbia(塞尔维亚人工智能研究与发展研究所)
Coding for Computation: Efficient Compression of Neural Networks for Reconfigurable Hardware
Institute for Digital Communications, Friedrich-Alexander-Universität (FAU), Erlangen, Germany(弗里德里希-亚历山大-埃朗根-纽伦堡大学数字通信研究所);Communications Engineering Lab, Karlsruhe Institute of Technology (KIT), Karlsruhe, Germany(卡尔斯鲁厄理工学院通信工程实验室);Electrical and Computer Engineering Department, University of Toronto, Canada(多伦多大学电气与计算机工程系)
University of California San Diego(加州大学圣地亚哥分校);ETH Zurich(苏黎世联邦理工学院);Fermi National Accelerator Laboratory(费米国家加速器实验室);University of Illinois Chicago(伊利诺伊大学芝加哥分校)
University of Edinburgh(爱丁堡大学);Microsoft Research(微软研究院);UC Irvine(加州大学尔湾分校);University of Washington(华盛顿大学);University of Alberta(阿尔伯塔大学)
我们考虑深度卷积网络(CNN)是否可以以与LSTM等递归网络相似的精度表示决策函数。首先,我们展示了一个深度CNN,其架构受到最近在图像识别中引入的模型的启发,可以在标准309 h Switchboard自动语音识别任务中产生比以前的卷积和LSTM网络更好的准确性。然后,我们证明了在LSTM的指导下,可以使用模型压缩的变体来训练更准确的CNN,我们称之为模型混合,因为教师和学生模型在复杂性上相似,但在归纳偏差上不同。混合进一步提高了我们的CNN的准确性,产生了一个计算效率高的模型,其准确性高于任何其他单独的模型。在这个模型压缩任务中检查“暗知识”的效果,我们发现,精确的模型压缩需要不到1%的最高概率标签。
Efficient Recurrent Neural Networks using Structured Matrices in FPGAs
Department of Electrical Engineering and Computer Science, Syracuse University, USA(Syracuse大学电气工程与计算机科学系);Center for Energy-efficient Computing and Applications (CECA), Peking University, China(北京大学能源高效计算与应用中心)
Compressing Recurrent Neural Networks for FPGA-accelerated Implementation in Fluorescence Lifetime Imaging
Center for Modeling, Simulation, & Imaging in Medicine(医学建模、仿真与成像中心);Rensselaer Polytechnic Institute(伦斯勒理工学院);IBM T.J. Watson Research Center(IBM TJ·沃森研究中心)
Ultra Memory-Efficient On-FPGA Training of Transformers via Tensor-Compressed Optimization
Department of Electrical and Computer Engineering, University of California, Santa Barbara(加州大学圣芭芭拉分校电子与计算机工程系);Intel Corporation(英特尔公司);Department of Computer Science, North Carolina State University(北卡罗来纳州立大学计算机科学系);School of Electrical and Computer Engineering, Georgia Institute of Technology(佐治亚理工学院电气与计算机工程学院)
CirCNN: Accelerating and Compressing Deep Neural Networks Using Block-CirculantWeight Matrices
Syracuse University(雪城大学);City University of New York, City College(纽约市立大学城市学院);University of Southern California(南加州大学);California State University Fullerton(加州州立大学富勒顿分校);Northeastern University(东北大学)