通过双delta测试评估混合精度计算的数值精度
Evaluating Numerical Accuracy in Mixed-Precision Computing by Dual-Delta Testing
AI总结:
本文提出双delta测试方法,用于评估混合精度计算中数值精度的准确性,通过对比高精度参考实现严格验证。
摘要:
混合精度计算在现代高性能计算和机器学习应用中变得越来越重要。当实现自定义的混合精度函数--如融合运算、优化的GPU内核或量化推理路径时,验证其数值精度至关重要。传统方法通常通过单一误差度量将自定义实现与参考进行比较。然而,这种单delta方法无法确定观察到的误差是由于精度级别本身还是特定实现导致的。本文介绍了双delta测试,一种系统的方法,通过与高精度oracle对比两个误差分布,使自定义实现与基线参考之间进行严格比较。我们提出了数学框架、算法公式、统计分析技术以及实际示例,展示了该方法在评估数值精度方面的有效性。
英文摘要:
Mixed-precision computing has become increasingly important in modern high-performance computing and machine learning applications. When implementing custom mixed-precision functions -- such as fused operators, optimized GPU kernels, or quantized inference paths -- it is critical to verify their numerical accuracy. Traditional approaches typically compare the custom implementation against a reference using a single error metric. However, this single-delta approach provides limited insight into whether the observed errors are inherent to the precision level or specific to the implementation. This paper introduces \textit{Dual-Delta Testing}, a systematic methodology that evaluates two error distributions against a high-precision oracle, enabling rigorous comparison between a custom implementation and a baseline reference. We present the mathematical framework, algorithmic formulation, statistical analysis techniques, and practical examples demonstrating the methodology's effectiveness in evaluating numerical accuracy.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。