Non-linear operations such as GELU, Layer normalization, and Softmax are essential yet costly building blocks of Transformer models. Several prior works simplified these operations with look-up tables or integer computations, but such approximations suffer inferior accuracy or considerable hardware cost with long latency. This paper proposes an accurate and hardware-friendly approximation framework for efficient Transformer inference. Our framework employs a simple neural network as a universal approximator with its structure equivalently transformed into a LUT. The proposed framework called NN-LUT can accurately replace all the non-linear operations in popular BERT models with significant reductions in area, power consumption, and latency.
Genetic Quantization-Aware Approximation for Non-Linear Operations in Transformers
The Hong Kong University of Science and Technology(香港科技大学);AI Chip Center for Emerging Smart Systems (ACCESS)(新兴智能系统AI芯片中心);Zhejiang University(浙江大学);Southern University of Science and Technology(南方科技大学)
HGQ-LUT: Fast LUT-Aware Training and Efficient Architectures for DNN Inference
California Institute of Technology, Pasadena, CA, USA(加州理工学院);University of Bristol, UK(布里斯托大学);Imperial College London, UK(伦敦帝国理工学院);SLAC National Accelerator Laboratory, Menlo Park, CA, USA(SLAC国家加速器实验室)
LL-ViT: Edge Deployable Vision Transformers with Look Up Table Neurons
Department of Electrical and Computer Engineering, The University of Texas at Austin(电气与计算机工程系,德克萨斯大学奥斯汀分校);The University of Texas at San Antonio(德克萨斯大学圣安东尼奥分校);Federal University of Rio de Janeiro(里约热内卢联邦大学);Instituto de Telecomunicações(电信研究所)
LUT Tensor Core: A Software-Hardware Co-Design for LUT-Based Low-Bit LLM Inference
Imperial College London(帝国理工学院伦敦分校);Microsoft Research London, UK(微软研究院伦敦, UK);Peking University(北京大学);Microsoft Research Beijing, China(微软研究院北京, China);University of Science(科学大学);University of Washington(华盛顿大学);Microsoft Research Seattle, USA(微软研究院西雅图, USA);Shanghai Jiao Tong University Shanghai, China(上海交通大学上海, China);Microsoft Research(微软研究院);Shanghai Jiao Tong University(上海交通大学)
Shrinking the Giant : Quasi-Weightless Transformers for Low Energy Inference
The University of Texas at Austin(德克萨斯大学奥斯汀分校);NVIDIA Research(英伟达研究院);The University of Texas at San Antonio(德克萨斯大学圣安东尼奥分校);IBM T.J. Watson Research Center(IBM T.J.沃森研究中心);Federal University of Rio de Janeiro(里约热内卢联邦大学);Universidade do Porto(波尔图大学);Instituto de Telecomunicações(电信研究所)
Multiobjective Hydropower Reservoir Operation Optimization with Transformer-Based Deep Reinforcement Learning
College of Computer Science and Technology, Nanjing University of Aeronautics and Astronautics(南京航空航天大学计算机科学与技术学院);Deptartment of Electrical Engineering and Computer Science, Lassonde School of Engineering, York University(约克大学拉桑德工程学院电气与计算机工程系)
Bitformer: An efficient Transformer with bitwise operation-based attention for Big Data Analytics at low-cost low-precision devices
University of Chinese Academy of Sciences(中国科学院大学);Shanghai Advanced Research Institute, Chinese Academy of Sciences(中国科学院上海高等研究院);Aston Business School, Aston University(阿斯顿大学阿斯顿商学院)
On the Expressive Power of Floating-Point Transformers
Department of Artificial Intelligence, Korea University(韩国大学人工智能系);Department of Mathematics and Statistics, Sejong University(世宗大学数学与统计学系);Department of Mathematical Sciences, Gwangju Institute of Science and Technology(全州科学技术研究院数学科学系)
Carnegie Mellon University(卡内基梅隆大学);Hong Kong University of Science and Technology(香港科技大学);Mohamed bin Zayed University of Artificial Intelligence(穆罕默德·本·扎耶德人工智能大学);Reality Labs, Meta Inc.(Meta公司Reality Labs)
MP-GFormer: A 3D-Geometry-Aware Dynamic Graph Transformer Approach for Machining Process Planning
School of Computing and Augmented Intelligence Arizona State University(计算与增强智能学院亚利桑那州立大学);Siemens Foundational Technologies(西门子基础技术);School of Manufacturing Systems and Networks Arizona State University(制造系统与网络学院亚利桑那州立大学)
Accurate typhoon intensity forecasts using a non-iterative spatiotemporal transformer model
National Meteorological Centre(国家气象中心);State Key Laboratory of Severe Weather, Chinese Academy of Meteorological Sciences, China Meteorological Administration(中国气象科学研究院灾害天气国家重点实验室,中国气象局)