In this work, we propose to quantize all parts of standard classification networks and replace the activation-weight--multiply step with a simple table-based lookup. This approach results in networks that are free of floating-point operations and free of multiplications, suitable for direct FPGA and ASIC implementations. It also provides us with two simple measures of per-layer and network-wide compactness as well as insight into the distribution characteristics of activationoutput and weight values. We run controlled studies across different quantization schemes, both fixed and adaptive and, within the set of adaptive approaches, both parametric and model-free. We implement our approach to quantization with minimal, localized changes to the training process, allowing us to benefit from advances in training continuous-valued network architectures. We apply our approach successfully to AlexNet, ResNet, and MobileNet. We show results that are within 1.6% of the reported, non-quantized performance on MobileNet using only 40 entries in our table. This performance gap narrows to zero when we allow tables with 320 entries. Our results give the best accuracies among multiply-free networks.
An Automata-Theoretic Approach to Synthesizing Binarized Neural Networks
National University of Defense Technology(国防科技大学);ShanghaiTech University(上海科技大学);Institute of Software, Chinese Academy of Sciences(中国科学院软件研究所);University of Chinese Academy of Sciences(中国科学院大学);Automotive Software Innovation Center(汽车软件创新中心)
Weightless Neural Networks for Efficient Edge Inference
The University of Texas at Austin(德克萨斯大学奥斯汀分校);Federal University of Recôncavo da Bahia(巴伊亚州雷孔卡沃联邦大学);Federal University of Rio de Janeiro(里约热内卢联邦大学);Universidade Federal Fluminense(弗鲁米嫩塞联邦大学);ISCTE Instituto Universitario de Lisboa(里斯本大学学院)
TsetlinWiSARD: On-Chip Training of Weightless Neural Networks using Tsetlin Automata on FPGAs
Institute for Clarity in Documentation(清晰文档研究所);Inria Paris-Rocquencourt(巴黎-罗克琴库特研究所);Rajiv Gandhi University(拉吉夫·甘地大学);Tsinghua University(清华大学);Palmer Research Laboratories(帕勒实验室)
LUT-KAN: Segment-wise LUT Quantization for Fast KAN Inference
Department of Theoretical and Applied Sciences (DiSTA)(理论与应用科学系);eCampus University(eCampus大学);Department of Intelligent Software Systems and Technologies(智能软件系统与技术系);School of Computer Science and Artificial Intelligence(计算机科学与人工智能学院);V.N. Karazin Kharkiv National University(V.N.卡扎林赫尔辛基国立大学)