arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2409.05207cs.LG
收藏

使用 hls4ml 在 FPGA 上实现面向物理应用的低延迟 Transformer 推理

Low Latency Transformer Inference on FPGAs for Physics Applications with hls4ml

University of Washington(华盛顿大学);Massachusetts Institute of Technology(麻省理工学院)

Zhixing Jiang, Dennis Yin, Yihui Chen, Elham E Khoda, Scott Hauck, Shih-Chieh Hsu, Ekaterina Govorkova, Philip Harris, Vladimir Loncar, Eric A. Moreno

AI总结:

提出基于 hls4ml 的 FPGA Transformer 部署方案,实现注意力、softmax 与归一化层,在 VU13P 上达到低于 2 μs 延迟并支持 TensorFlow 模型。

摘要:

本研究利用 hls4ml 在现场可编程门阵列(FPGA)中高效实现 Transformer 架构。我们展示了实现多头注意力、softmax 和归一化层的策略,并评估了三个不同模型。将它们部署在 VU13P FPGA 芯片上后,延迟低于 2 μs,证明了其用于实时应用的潜力。hls4ml 与任何由 TensorFlow 构建的 Transformer 模型兼容,进一步增强了本工作的可扩展性和适用性。索引词:FPGA、机器学习、Transformer、高能物理、LIGO。

英文摘要:

This study presents an efficient implementation of transformer architectures in Field-Programmable Gate Arrays(FPGAs) using hls4ml. We demonstrate the strategy for implementing the multi-head attention, softmax, and normalization layer and evaluate three distinct models. Their deployment on VU13P FPGA chip achieved latency less than 2us, demonstrating the potential for real-time applications. HLS4ML compatibility with any TensorFlow-built transformer model further enhances the scalability and applicability of this work. Index Terms: FPGAs, machine learning, transformers, high energy physics, LIGO

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑