arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2005.14187cs.CLcs.LGcs.NE
收藏

HAT:面向高效自然语言处理的硬件感知 Transformer

HAT: Hardware-Aware Transformers for Efficient Natural Language Processing

Massachusetts Institute of Technology(麻省理工学院);MIT-IBM Watson AI Lab(麻省理工学院-IBM沃森人工智能实验室)

Hanrui Wang, Zhanghao Wu, Zhijian Liu, Han Cai, Ligeng Zhu, Chuang Gan, Song Han

AI总结:

本文提出 HAT,通过神经架构搜索、SuperTransformer 权重共享与延迟约束进化搜索,为 CPU、GPU 和 IoT 硬件自动发现高效 Transformer。

摘要:

Transformer 在自然语言处理(NLP)任务中无处不在,但由于计算密集,它们难以部署到硬件上。为了在资源受限的硬件平台上实现低延迟推理,我们提出利用神经架构搜索来设计硬件感知 Transformer(Hardware-Aware Transformers,HAT)。我们首先构建一个包含 arbitrary encoder-decoder attention(任意编码器-解码器注意力)和 heterogeneous layers(异构层)的大型设计空间。随后,我们训练一个覆盖该设计空间中所有候选结构的 SuperTransformer,并通过权重共享高效生成许多 SubTransformer。最后,我们在硬件延迟约束下执行进化搜索,以找到一个专门用于在目标硬件上快速运行的 SubTransformer。在四个机器翻译任务上的大量实验表明,HAT 能够为不同硬件(CPU、GPU、IoT 设备)发现高效模型。在 Raspberry Pi-4 上运行 WMT'14 翻译任务时,与基线 Transformer 相比,HAT 可实现 3 倍加速、体积缩小 3.7 倍;与 Evolved Transformer 相比,可实现 2.7 倍加速、体积缩小 3.6 倍,同时搜索成本降低 12,041 倍且没有性能损失。HAT 代码位于 https://github.com/mit-han-lab/hardware-aware-transformers.git。

英文摘要:

Transformers are ubiquitous in Natural Language Processing (NLP) tasks, but they are difficult to be deployed on hardware due to the intensive computation. To enable low-latency inference on resource-constrained hardware platforms, we propose to design Hardware-Aware Transformers (HAT) with neural architecture search. We first construct a large design space with $\textit{arbitrary encoder-decoder attention}$ and $\textit{heterogeneous layers}$. Then we train a $\textit{SuperTransformer}$ that covers all candidates in the design space, and efficiently produces many $\textit{SubTransformers}$ with weight sharing. Finally, we perform an evolutionary search with a hardware latency constraint to find a specialized $\textit{SubTransformer}$ dedicated to run fast on the target hardware. Extensive experiments on four machine translation tasks demonstrate that HAT can discover efficient models for different hardware (CPU, GPU, IoT device). When running WMT'14 translation task on Raspberry Pi-4, HAT can achieve $\textbf{3}\times$ speedup, $\textbf{3.7}\times$ smaller size over baseline Transformer; $\textbf{2.7}\times$ speedup, $\textbf{3.6}\times$ smaller size over Evolved Transformer with $\textbf{12,041}\times$ less search cost and no performance loss. HAT code is https://github.com/mit-han-lab/hardware-aware-transformers.git

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑