当量化破坏内存：低精度时序推理中的循环状态写回机制
When Quantization Breaks Memory: Recurrent-State Write-Back in Low-Precision Temporal Inference
浏览论文内容
中文总结 AI 辅助
本文提出循环状态写回概念，发现低精度量化会导致循环网络估计荧光寿命参数的误差大幅上升，而误差反馈等机制可恢复精度，状态存储接口是量化循环推理的核心设计考量。
中文摘要 AI 辅助
量化技术被广泛用于降低神经网络推理的计算与内存需求。然而在循环网络中，量化后的状态会被存储并在下一时间步返回，因此用于存储该状态的规则会改变后续计算。本文引入「循环状态写回」这一术语来指代该规则，并在用于定量生物成像的分子成像模态——荧光寿命成像的紧凑GRU编解码器中分离其影响。核心任务是从高噪声时间分辨荧光信号中估计两个寿命参数：短寿命组分τ₁和长寿命组分τ₂。保持训练好的模型固定，用确定性4位状态存储替代连续状态传播，会使τ₁和τ₂的估计误差分别增加约70倍和300倍。当重复的微小更新低于写回阈值时，就会出现失效，此时存储的状态几乎固定，而网络仍在提出改变。误差反馈、残余记忆和方向记忆会在时间维度上传送这些被抑制更新的信息，无需重新训练即可恢复精度。精度扫描显示，提高状态精度可能会恶化固定的循环解，而匹配训练表明可以学习与状态接口的兼容性。为测试该行为是否超出GRU范围，我们在独立训练的LSTM中重复训练后干预，结果显示粗粒度写回会重现失效现象，误差反馈可恢复精度，且状态特定干预揭示了细胞状态比隐藏状态具有更高的敏感性。我们的研究确立循环状态写回是低精度循环动力学的关键决定因素，并确定状态存储接口是量化循环推理的核心设计考量。
英文摘要
Quantization is widely used to reduce the computational and memory demands of neural-network inference. In recurrent networks, however, the quantized state is stored and returned at the next time step, so the rule used to store that state can alter subsequent computations. Here, we introduce recurrent-state write-back to denote this rule and isolate its effect in a compact GRU encoder--decoder for fluorescence lifetime imaging, a molecular imaging modality used in quantitative biological imaging. A central task is estimating two lifetime parameters, the short-lived component τ1 and the long-lived component τ2, from high-noise time-resolved fluorescence signals. Holding the trained model fixed, replacing continuous state propagation with deterministic 4-bit state storage increases estimation errors for τ1 and τ2 by approximately 70x and 300x, respectively. Failure occurs when repeated small updates remain below the write threshold, leaving the stored state nearly fixed while the network continues to propose change. Error feedback, residual memory, and direction memory carry information from these suppressed updates across time and recover accuracy without retraining. Precision sweeps show that increasing state precision can worsen a fixed recurrent solution, while matched training shows that compatibility with the state interface can be learned. To test whether this behavior extends beyond the GRU, we repeat the post-training intervention in an independently trained LSTM, where coarse write-back reproduces the failure, error feedback restores accuracy, and state-specific interventions reveal greater sensitivity of the cell state than the hidden state. Our results establish recurrent-state write-back as a key determinant of low-precision recurrent dynamics and identify the state-storage interface as a central design consideration for quantized recurrent inference.
发表机构
- Rensselaer Polytechnic Institute（伦斯勒理工学院）
- Center for Modeling, Simulation, and Imaging in Medicine（医学建模、仿真与成像中心）
- Department of Biomedical Engineering（生物医学工程系）
机构由 AI 辅助整理，请以论文原文为准。
