预测编码网络中隐藏状态优化顺序的研究
A Study of Hidden-State Optimization Order in Predictive Coding Networks
浏览论文内容
中文总结 AI 辅助
该研究针对局部学习方法特征学习弱的问题，提出边界优先推理方案并实例化为预测编码网络，在CIFAR-10上提升了准确率，验证了该方案的有效性。
中文摘要 AI 辅助
局部学习方法为端到端反向传播提供了替代方案，但其非结构化的局部目标可能导致深度网络的特征学习效果较弱。本研究探讨隐藏状态的优化顺序是否可解决这一局限。我们提出一种边界优先推理方案，该方案将模型划分为多个块，首先协调块边界处的隐藏状态，随后优化每个块内的表示。我们将该方案实例化为预测编码网络（PCNs，一种在推理过程中明确暴露隐藏活动与预测误差的局部学习框架）。在CIFAR-10数据集上，所得的边界优先预测编码实例在标准参数设置下，较标准预测编码的准确率提升9.77%；在μ参数设置下，准确率提升5.51%。诊断分析进一步显示，该方案具有更多非平凡的早期层更新、更低的初始至最终层的中心核对齐（CKA），以及更多样化的分层梯度，这些均与更强的特征学习效果一致。上述结果表明，基于块的边界优先推理是预测编码训练的实用设计原则，并推动其在更广泛的局部学习系统中的研究。
英文摘要
Local learning methods offer an alternative to end-to-end backpropagation, but their unstructured local objectives can produce weak feature learning in deep networks. We study whether the order of hidden-state optimization can address this limitation. We propose a boundary-first inference schedule that partitions a model into chunks, first coordinates hidden states at chunk boundaries, and then refines representations within each chunk. We instantiate this schedule in predictive coding networks (PCNs), a local-learning framework in which hidden activities and prediction errors are explicitly exposed during inference. On CIFAR-10, the resulting boundary-first predictive-coding instantiation improves accuracy over standard predictive coding by $9.77\%$ under a standard parametrization and by $5.51\%$ under a $μ$-parametrization. Diagnostic analyses further show more non-trivial early-layer updates, lower initial-to-final CKA, and more diverse layerwise gradients, consistent with stronger feature learning. These results support boundary-first, chunk-based inference as a practical design principle for predictive-coding training and motivate its study in broader local-learning systems.
