TriGen:基于软硬件协同设计的NPU架构,用于大型语言模型的端到端加速
TriGen: NPU Architecture for End-to-End Acceleration of Large Language Models based on SW-HW Co-Design
AI总结:
TriGen通过软硬件协同设计,提出了一种针对资源受限环境的NPU架构,实现大型语言模型的端到端加速,平均性能提升2.73倍,内存传输减少52%。
摘要:
近期研究表明,NPU架构在设备端加速AI推理方面已被广泛研究,这些架构本质上资源受限。同时,基于transformer的大型语言模型(LLMs)已成为主流,尽管模型规模迅速增加,但与传统CNN相比,参数复用程度较低,使得在资源受限的设备上实现端到端执行极为具有挑战性。为了解决这些挑战,我们提出了TriGen,一种针对资源受限环境设计的新型NPU架构,通过软硬件协同设计。首先,TriGen采用低精度计算利用微缩(MX)技术,在保持精度的同时实现额外的优化机会,并解决由此产生的问题。其次,为了联合优化非线性和线性操作,TriGen通过使用快速且准确的LUT消除了对非线性操作专用硬件的需求,从而最大化性能提升并减少设备环境中的硬件成本。最后,通过考虑实际硬件限制,进一步采用调度技术以在有限的芯片内存容量下最大化计算利用率。我们在各种LLMs上评估了TriGen的性能,并展示TriGen在基准NPU设计上实现了平均2.73倍的性能提升,并且内存传输减少了52%,同时精度损失极小。
英文摘要:
Recent studies have extensively explored NPU architectures for accelerating AI inference in on-device environments, which are inherently resource-constrained. Meanwhile, transformer-based large language models (LLMs) have become dominant, with rapidly increasing model sizes but low degree of parameter reuse compared to conventional CNNs, making end-to-end execution on resource-limited devices extremely challenging. To address these challenges, we propose TriGen, a novel NPU architecture tailored for resource-constrained environments through software-hardware co-design. Firstly, TriGen adopts low-precision computation using microscaling (MX) to enable additional optimization opportunities while preserving accuracy, and resolves the issues that arise by employing such precision. Secondly, to jointly optimize both nonlinear and linear operations, TriGen eliminates the need for specialized hardware for essential nonlinear operations by using fast and accurate LUT, thereby maximizing performance gains and reducing hardware-cost in on-device environments, and finally, by taking practical hardware constraints into account, further employs scheduling techniques to maximize computational utilization even under limited on-chip memory capacity. We evaluate the performance of TriGen on various LLMs and show that TriGen achieves an average 2.73x performance speedup and 52% less memory transfer over the baseline NPU design with negligible accuracy loss.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。