arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2112.02191cs.LGcs.AI
收藏

NN-LUT:用于有效Transformer推理的非线性运算的神经近似

NN-LUT: Neural Approximation of Non-Linear Operations for Efficient Transformer Inference

NAVER AI Lab(NAVER人工智能实验室);NAVER Clova;SAIT(三星高级技术研究院);Hanyang University(汉阳大学)

Joonsang Yu, Junki Park, Seongmin Park, Minsoo Kim, Sihwa Lee, Dong Hyun Lee, Jungwook Choi

摘要:

GELU、层归一化和Softmax等非线性操作是Transformer模型的基本但昂贵的构建块。一些现有的工作简化了这些操作与查找表或整数计算,但这样的近似遭受较差的精度或相当大的硬件成本与长的延迟。本文提出了一个准确的和硬件友好的近似框架,有效的Transformer推理。我们的框架采用了一个简单的神经网络作为一个通用的逼近器,其结构等价地转换成一个LUT。提出的框架称为NN-LUT可以准确地取代流行的BERT模型中的所有非线性操作,并显着减少面积,功耗和延迟。

英文摘要:

Non-linear operations such as GELU, Layer normalization, and Softmax are essential yet costly building blocks of Transformer models. Several prior works simplified these operations with look-up tables or integer computations, but such approximations suffer inferior accuracy or considerable hardware cost with long latency. This paper proposes an accurate and hardware-friendly approximation framework for efficient Transformer inference. Our framework employs a simple neural network as a universal approximator with its structure equivalently transformed into a LUT. The proposed framework called NN-LUT can accurately replace all the non-linear operations in popular BERT models with significant reductions in area, power consumption, and latency.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑