发表机构
Sorbonne Université; CNRS; LIP6（索邦大学; 法国国家科学研究中心; 巴黎第六大学信息学实验室）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
研究深度学习算子数值稳定性问题，核心方法是集成CESTAC的统一软件工具，贡献在于能单次计算验证、检测不稳定源及提供监测，经实验验证有效，为开发数值稳定计算内核提供见解。
AI 中文摘要
有限精度算术不可避免地会引入数值近似误差，数值计算可能因精度不足或公式不当导致数值不稳定。本文介绍了首个集成CESTAC的统一软件工具，用于检测深度学习算子的数值稳定性。该软件不仅能单次计算验证数值，还能检测数值不稳定源，并在深度学习训练和推理时提供稳定性监测。通过在各种任务中对注入数值不稳定的污染算子检测，验证了其有效性。该方法和工具为开发数值稳定的计算内核提供了有价值的见解，对深度学习训练和推理的数值稳定性和效率至关重要。
英文摘要
Finite-precision arithmetic unavoidably introduces numerical approximation errors. Numerical computations may use insufficient precision or an improper formulation, which leads to numerical instability. In this paper, we introduce a unified software tool for stochastic numerical validation of deep-learning operators. The tool follows CESTAC on supported exposed operations and uses an operator-level data-perturbation approximation for GEMM-like kernels. Our developed software not only enables numerical validation with a single computation pass but also detects the sources of numerical instability and provides numerical stability monitoring during deep learning training and inference. We verified its effectiveness on the detection of polluted operators with injected numerical instabilities across various tasks. We believe that our developed method and tools provide valuable insights into developing numerically stable computing kernels, which are particularly critical for numerically stable and efficient deep learning training and inference.
