The emergence of neural network capabilities invariably leads to a significant surge in computational demands due to expanding model sizes and increased computational complexity. To reduce model size and lower inference costs, recent research has focused on simplifying models and designing hardware accelerators using low-bit quantization. However, due to numerical representation limits, scalar quantization cannot reduce bit width lower than 1-bit, diminishing its benefits. To break through these limitations, we introduce LUT-DLA, a Look-Up Table (LUT) Deep Learning Accelerator Framework that utilizes vector quantization to convert neural network models into LUTs, achieving extreme low-bit quantization. The LUT-DLA framework facilitates efficient and cost-effective hardware accelerator designs and supports the LUTBoost algorithm, which helps to transform various DNN models into LUT-based models via multistage training, drastically cutting both computational and hardware overhead. Additionally, through co-design space exploration, LUT-DLA assesses the impact of various model and hardware parameters to fine-tune hardware configurations for different application scenarios, optimizing performance and efficiency. Our comprehensive experiments show that LUT-DLA achieves improvements in power efficiency and area efficiency with gains of $1.4$~$7.0\times$ and $1.5$~$146.1\times$, respectively, while maintaining only a modest accuracy drop. For CNNs, accuracy decreases by $0.1\%$~$3.1\%$ using the $L_2$ distance similarity, $0.1\%$~$3.4\%$ with the $L_1$ distance similarity, and $0.1\%$~$3.8\%$ when employing the Chebyshev distance similarity. For transformer-based models, the accuracy drop ranges from $1.4\%$ to $3.0\%$.
From Arithmetic to Logic: The Resilience of Logic and Lookup-Based Neural Networks Under Parameter Bit-Flips
The University of Texas at Austin(德克萨斯大学奥斯汀分校);Federal University of Rio de Janeiro(里约热内卢联邦大学);Instituto de Telecomunicações, Porto, Portugal(葡萄牙波尔图电信研究所);Google LLC(谷歌公司)
HGQ-LUT: Fast LUT-Aware Training and Efficient Architectures for DNN Inference
California Institute of Technology, Pasadena, CA, USA(加州理工学院);University of Bristol, UK(布里斯托大学);Imperial College London, UK(伦敦帝国理工学院);SLAC National Accelerator Laboratory, Menlo Park, CA, USA(SLAC国家加速器实验室)
LUT Tensor Core: A Software-Hardware Co-Design for LUT-Based Low-Bit LLM Inference
Imperial College London(帝国理工学院伦敦分校);Microsoft Research London, UK(微软研究院伦敦, UK);Peking University(北京大学);Microsoft Research Beijing, China(微软研究院北京, China);University of Science(科学大学);University of Washington(华盛顿大学);Microsoft Research Seattle, USA(微软研究院西雅图, USA);Shanghai Jiao Tong University Shanghai, China(上海交通大学上海, China);Microsoft Research(微软研究院);Shanghai Jiao Tong University(上海交通大学)
ALLMod: Exploring $\underline{\mathbf{A}}$rea-Efficiency of $\underline{\mathbf{L}}$UT-based $\underline{\mathbf{L}}$arge Number $\underline{\mathbf{Mod}}$ular Reduction via Hybrid Workloads