深度何时能取代精度？量化神经计算的资源理论
When Can Depth Replace Precision? A Resource Theory of Quantized Neural Computation
查看机构详情
- University of Illinois Chicago（伊利诺伊大学芝加哥分校）
机构由 AI 辅助整理，请以论文原文为准。
浏览论文内容
中文总结 AI 辅助
研究固定输入输出映射下低比特残差计算能否取代数值精度，通过建模量化残差系统刻画无限深度极限，得出纯调度接近松弛类的速率等结论，还探讨了多种相关情况，指出深度取代精度有条件限制。
中文摘要 AI 辅助
我们研究了对于固定的输入-输出映射，额外的低比特残差计算何时能取代缺失的数值精度。将固定时间范围内的量化残差系统建模为从声明的低比特操作库中选择字段的纯调度，并使用松弛控制来刻画其无限深度极限。得出了纯调度在不同时间依赖下接近松弛类的速率，执行算法会改变结论，还研究了固定教师情况、学习码本和状态依赖路由等情况。验证的原对偶界在训练前给出可行、不可行或未解决的决策。深度仅相对于声明的库、时间范围、执行语义和路由模型才能取代精度。
英文摘要
When can additional low-bit residual computation replace missing numerical precision for a fixed input-output map? We model a quantized residual system over a fixed horizon as a pure schedule selecting fields from a declared low-bit operation library, and use relaxed controls to characterize its infinite-depth limit. The distance from the target to the closed relaxed reachable set is the exact structural floor: no increase in depth can remove it for that library. Pure schedules approach the relaxed class at rate $O(D^{-1})$ under bounded-variation time dependence and $O(D^{-\vartheta}+D^{-1})$ under Holder dependence of exponent $\vartheta$. Execution arithmetic can reverse this conclusion: full-state write-back introduces a $Dρ_z$ penalty and can freeze residual updates, whereas increment error feedback replaces this growth by a bounded carry term and obeys an exact common-lattice conservation law. A fixed-teacher converse makes this rate sharp: for coherent depth-$L$ first-order high-precision comparators, accuracy matching requires $D=Θ(L)$. Learned codebooks add a metadata resource, while state-dependent routing introduces hybrid event conditions. Verified primal and dual bounds yield feasible, impossible, or unresolved decisions before training. Companion software implements the workflow, and Lean 4 machine-checks the exact discrete core. Depth replaces precision only relative to a declared library, horizon, execution semantics, and routing model.
