从噪声调制神经网络的前向波动中重构反向传播算法
Reconstructing Backpropagation from Forward Fluctuations in Noise-modulated Neural Networks
浏览论文内容
中文总结 AI 辅助
该研究针对噪声调制神经网络的权重传输问题，提出仅用前向传播统计重构反向传播的方法，其梯度接近无偏，在简单回归任务上精度与反向传播相当，适配数字电路。
中文摘要 AI 辅助
噪声调制神经网络（NNN）仅在存在噪声时进行学习与推理，将噪声视为计算资源而非干扰。该网络借助噪声可通过反向传播高效学习，同时传输类尖峰信号，但反向传播需通过转置权重的反向路径，即权重传输问题，这削弱了其生物与神经形态合理性。仅前向传播的替代方案通常会替换不同目标或固定随机反馈，牺牲了稳定性与准确性。我们证明，在NNN中仅通过前向传播统计即可重构反向传播：权重镜像通过前一层单元输出与后一层单元输入的协方差估计各权重矩阵，结合单元内部的局部差分估计，沿计算图递归传播输出误差，无需转置权重读取与反向数据路径。所得梯度经验上接近无偏，结合逐权重的局部Adam更新，在简单回归任务上可达到反向传播的最终精度。采用均匀分布噪声时，局部操作可简化为多项式与比较器，使整个系统（包括学习规则）适配数字电路。因此，在NNN中，噪声不仅是推理的资源，也是重构反向传播的资源。
英文摘要
A Noise-modulated Neural Network (NNN) learns and infers only in the presence of noise, treating noise as a computational resource rather than a disturbance. The noise lets it learn efficiently by backpropagation while transmitting spike-like signals, but backpropagation needs a reverse path through transposed weights, the weight transport problem, which undermines biological and neuromorphic plausibility. Forward-only alternatives typically substitute a different objective or fixed random feedback, sacrificing stability and accuracy. We show that backpropagation itself can be reconstructed in the NNN from forward-pass statistics alone: a weight mirror estimates each weight matrix from the covariance between a previous-layer unit's output and the next-layer unit's input, and combining it with local differential estimation inside the units propagates the output error recursively along the computational graph, with no transposed-weight readout and no backward data path. The resulting gradient is empirically near-unbiased, and with local per-weight Adam updates it matches the final accuracy of backpropagation on simple regression tasks. With uniformly distributed noise, the local operations reduce to polynomials and comparators, making the whole system, learning rule included, well suited to digital circuits. Thus, in the NNN, noise is a resource not only for inference but also for reconstructing backpropagation.
