APT-LLM: 利用任意精度Tensor Core计算加速大语言模型
APT-LLM: Exploiting Arbitrary-Precision Tensor Core Computing for LLM Acceleration
AI总结：
针对大语言模型量化加速中GPU Tensor Core支持不足等问题，提出APT-LLM方案，通过新型数据格式、比特级矩阵乘法、优化内存管理及动态内核映射，实现多硬件平台上的显著推理加速。
摘要：
大语言模型（LLM）已彻底改变了AI应用，但其巨大的计算需求严重限制了部署和实时性能。量化方法有助于降低计算成本，然而，在GPU上实现任意精度的超低比特量化LLM的极致效率仍面临挑战，这主要源于GPU Tensor Core支持有限、内存管理效率低下以及内核优化缺乏灵活性。为解决这些挑战，我们提出了一套面向任意精度LLM的综合加速方案，即APT-LLM。首先，我们提出了一种新型数据格式bipolar-INT，它可与有符号INT实现高效无损转换，同时更有利于并行计算。我们还开发了一种支持任意精度的矩阵乘法（MatMul）方法，通过在比特层面拆分和重组矩阵实现，该方法提供了灵活的精度，并优化了GPU Tensor Core的利用率。此外，我们提出了一种以数据恢复为核心的内存管理系统，策略性地使用高速共享内存，大幅提升内核执行速度并降低内存访问延迟。最后，我们开发了一种内核映射方法，可针对不同矩阵大小动态选择内核的最优可配置超参数，从而在不同LLM架构和精度设置下实现最佳性能。在LLM推理中，在RTX 3090上，APT-LLM相比FP16基准实现了最高3.99倍的加速，相比NVIDIA CUTLASS INT4加速实现了2.16倍的加速；在RTX 4090和H800上，APT-LLM相比FP16实现了最高2.44倍的加速，相比CUTLASS整数基准实现了1.65倍的加速。
英文摘要：
Large language models (LLMs) have revolutionized AI applications, yet their enormous computational demands severely limit deployment and real-time performance. Quantization methods can help reduce computational costs, however, attaining the extreme efficiency associated with ultra-low-bit quantized LLMs at arbitrary precision presents challenges on GPUs. This is primarily due to the limited support for GPU Tensor Cores, inefficient memory management, and inflexible kernel optimizations. To tackle these challenges, we propose a comprehensive acceleration scheme for arbitrary precision LLMs, namely APT-LLM. Firstly, we introduce a novel data format, bipolar-INT, which allows for efficient and lossless conversion with signed INT, while also being more conducive to parallel computation. We also develop a matrix multiplication (MatMul) method allowing for arbitrary precision by dismantling and reassembling matrices at the bit level. This method provides flexible precision and optimizes the utilization of GPU Tensor Cores. In addition, we propose a memory management system focused on data recovery, which strategically employs fast shared memory to substantially increase kernel execution speed and reduce memory access latency. Finally, we develop a kernel mapping method that dynamically selects the optimal configurable hyperparameters of kernels for varying matrix sizes, enabling optimal performance across different LLM architectures and precision settings. In LLM inference, APT-LLM achieves up to a 3.99$\times$ speedup compared to FP16 baselines and a 2.16$\times$ speedup over NVIDIA CUTLASS INT4 acceleration on RTX 3090. On RTX 4090 and H800, APT-LLM achieves up to 2.44$\times$ speedup over FP16 and 1.65$\times$ speedup over CUTLASS integer baselines.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
