基于浮点数分组共享指数的精度感知迭代算法
Precision-Aware Iterative Algorithms Based on Group-Shared Exponents of Floating-Point Numbers
AI总结:
本文提出基于分组共享指数和尾数分段存储的浮点表示及阶梯式混合精度迭代算法,解决SpMV内存瓶颈,显著提升性能与收敛残差。
摘要:
迭代求解器在科学应用和工程计算中经常使用。然而,受内存限制的稀疏矩阵向量(SpMV)核计算阻碍了迭代算法的效率。随着现代硬件越来越支持低精度计算,迭代算法的混合精度优化引起了广泛关注。尽管如此,现有的混合精度方法带来了挑战,包括格式转换开销、存储与计算表示之间的紧密耦合,以及需要存储数据的多个精度副本。本文提出了一种基于分组共享指数和尾数分段存储的浮点数表示方法,实现了表示向量更高的比特利用率,并且无需多个数据副本即可在不同精度之间快速切换。此外,提出了一种阶梯式混合精度迭代算法。我们的实验结果表明,与现有的浮点格式相比,我们的方法显著提高了迭代算法的性能和收敛残差。
英文摘要:
Iterative solvers are frequently used in scientific applications and engineering computations. However, the memory-bound Sparse Matrix-Vector (SpMV) kernel computation hinders the efficiency of iterative algorithms. As modern hardware increasingly supports low-precision computation, the mixed-precision optimization of iterative algorithms has garnered widespread attention. Nevertheless, existing mixed-precision methods pose challenges, including format conversion overhead, tight coupling between storage and computation representation, and the need to store multiple precision copies of data. This paper proposes a floating-point representation based on the group-shared exponent and segmented storage of the mantissa, enabling higher bit utilization of the representation vector and fast switches between different precisions without needing multiple data copies. Furthermore, a stepped mixed-precision iterative algorithm is proposed. Our experimental results demonstrate that, compared with existing floating-point formats, our approach significantly improves iterative algorithms' performance and convergence residuals.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。