数值精度达到多少才算足够？
How Accurate Is Accurate Enough?
浏览论文内容
中文总结 AI 辅助
本文通过学习目标分析数值误差的影响，提出基于 softmax 交叉熵的有限误差保证与认证容差方法，证明数值精度需结合学习状态评估，并应纳入学习目标。
中文摘要 AI 辅助
在学习系统中，数值逼近必须达到多高的精度？仅凭原始误差本身无法回答这个问题：相同量级的误差在不同学习状态下，对损失、预测和梯度可能产生截然不同的后果。我们通过学习目标本身来研究这个问题。目标函数根据当前状态对类别级数值误差进行非均匀加权，因此误差的重要性不仅取决于其大小，还取决于其影响的类别以及该类别所获得的权重。对于 softmax 交叉熵，我们刻画了类别权重与误差之间的这种耦合关系，并在固定非目标概率和目标分数误差的多重集条件下，推导出固定非目标概率与分数误差配对下符号损失变化的精确极值，同时保持目标概率和目标分数误差不变。基于这一结构，我们建立了有限误差保证，将原始误差传播到损失、概率、预测和特征梯度，然后反转这些保证，在规定的学习级误差要求下，获得当前状态的认证原始容差。我们在高维 von Mises-Fisher 学习中给出了该框架的完整实例化。受控干预和大量保存的学习状态表明，相同的原始误差可以产生显著不同的学习后果，而在相同学习级要求下，认证数值容差在不同状态间相差多个数量级。这些结果表明，数值逼近的充分性必须结合当前学习状态和待保持的量来评估；数值精度本身应被视为学习目标的一部分。
英文摘要
How accurate must a numerical approximation be within a learning system? Primitive error alone cannot answer this question: errors of the same magnitude can have very different consequences for losses, predictions, and gradients at different learning states. We study this question through the learning objective itself. The objective weights classwise numerical errors nonuniformly according to the current state, so the importance of an error depends not only on its magnitude but also on the class it affects and the weight that class receives. For softmax cross-entropy, we characterize this coupling between class weights and errors and derive the exact extrema of the signed loss change over pairings of fixed non-target probability and score-error multisets, with the target probability and target score error held fixed. Building on this structure, we establish finite-error guarantees that propagate primitive error to losses, probabilities, predictions, and feature gradients, then invert these guarantees to obtain a certified primitive tolerance for the current state under prescribed learning-level error requirements. We give a complete instantiation of the framework in high-dimensional von Mises-Fisher learning. Controlled interventions and a large collection of saved learning states show that identical primitive error can produce substantially different learning consequences, while certified numerical tolerances vary by orders of magnitude across states under the same learning-level requirements. These results show that the adequacy of a numerical approximation must be assessed in relation to the current learning state and the quantity to be preserved; numerical accuracy should itself be treated as part of the learning objective.
发表机构
- Nanjing Normal University（南京师范大学）
- Nanjing University of Chinese Medicine（南京中医药大学）
机构由 AI 辅助整理，请以论文原文为准。
