arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2607.04531cs.LGcs.AIcs.NAeess.SPmath.NA
收藏

定点运算下硬件安全神经网络的李雅普诺夫引导训练

Lyapunov-Guided Training for Hardware-Safe Neural Networks Under Fixed-Point Arithmetic

The School of Engineering & Physical Sciences(工程与物理科学学院);SDAIA-KFUPM Joint Research Centre for Artificial Intelligence(SDAIA-KFUPM人工智能联合研究中心);Heriot-Watt University(赫瑞-沃森大学);King Fahd University of Petroleum and Minerals(国王法赫德石油大学)

Anis Hamadouche, Amir Hussain

AI总结:

针对定点运算中低精度神经网络的问题,提出李雅普诺夫稳定量化框架,通过层状李雅普诺夫函数监测隐藏状态能量,应用单调投影确保状态稳定,实验表明该方法可抑制激活溢出并恢复稳定学习。

摘要:

低精度神经网络对资源受限硬件有吸引力,但定点运算引入的故障模式常被理想化量化模型掩盖。本文为硬件式环绕运算下的低精度神经网络提出李雅普诺夫稳定量化框架,通过层状李雅普诺夫函数监测隐藏状态能量,并应用单调投影来强制跨深度的有界且非递增状态演化。该方法在MNIST上进行了评估,实验结果表明基于李雅普诺夫的状态控制可作为一种硬件感知稳定机制,用于可靠的定点神经推理和训练。

英文摘要:

Low-precision neural networks are attractive for resource-constrained hardware, but fixed-point arithmetic introduces failure modes that are often hidden by idealised quantisation models. In particular, two's-complement overflow wrapping can corrupt hidden activations by changing both their magnitude and sign, leading to unstable numerical error propagation and severe accuracy degradation. This paper proposes a Lyapunov-stabilised quantisation framework for low-precision neural networks operating under hardware-style wrapping arithmetic. The hidden-state energy is monitored through a layerwise Lyapunov function, and a monotone projection is applied to enforce bounded and non-increasing state evolution across depth. The method is evaluated on MNIST using a compact patch-based transformer under post-training quantisation and quantisation-aware training with fixed-point bit-widths from 4 to 16 bits. Monte Carlo results show that unconstrained wrapped quantisation-aware training collapses to near-chance accuracy across 6-16 bits, with activation overflow rates exceeding 11%. In contrast, the proposed monotone Lyapunov projection suppresses activation overflow to below 0.012% and restores stable low-precision learning, achieving 86.55% accuracy at 12 bits. These results demonstrate that Lyapunov-based state control can act as a hardware-aware stabilisation mechanism for reliable fixed-point neural inference and training.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑