arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2605.06240cs.LGcs.AI
收藏

累积良好性免费搭车在前向-前向网络中:真实、可修复,但不主导精度

Cumulative-Goodness Free-Riding in Forward-Forward Networks: Real, Repairable, but Not Accuracy-Dominant

Amirhossein Yousefiramandi(阿米尔霍西·尤塞菲拉曼迪)

Amirhossein Yousefiramandi

AI总结:

本文研究了前向-前向网络中累积良好性免费搭车现象,提出三种局部修复方法提升层间分离度,实验表明在CIFAR-10和CIFAR-100上深度层分离度显著提升,但对精度影响有限。

摘要:

前向-前向(FF)训练允许每层根据局部良好性标准学习。在累积良好性变体中,后续层可以继承先前层已部分分离的任务。我们正式将这种现象定义为层免费搭车:在软plus FF标准下,到达第d块的类间区分梯度随前序块积累的正边距指数衰减。随后研究了三种局部修复方法——按块、难度门控和深度缩放,这些方法在不依赖反向传播梯度的情况下恢复当前层分离度。在CIFAR-10和CIFAR-100上,这些修复方法显著提高了层分离统计量,深度层的提升幅度为4倍至45倍,而对精度的影响小于1个百分点(非退化训练过程)。Tiny ImageNet提供了一个更严苛的跨数据集检验,揭示了层健康诊断与最终精度之间的相同定性差距。校准实验进一步显示,架构和增强选择对最终精度的影响大于本文研究的训练规则修改。因此,累积免费搭车是一种真实且可修复的优化病理现象。然而,对于研究的FF训练规则、架构和数据集,它并非限制可实现精度的主要因素。

英文摘要:

Forward-Forward (FF) training lets each layer learn from a local goodness criterion. In cumulative-goodness variants, later layers can inherit a task that earlier layers have partly separated. We formalize this as layer free-riding: under the softplus FF criterion, the class-discrimination gradient reaching block $d$ decays exponentially with the positive margin accumulated by earlier blocks (Theorem 3.1); a squared-hinge barrier reproduces the pathology. History-free and hardness-gated repairs raise deeper-layer separation by up to $5\times$ on CIFAR-100 and $45\times$ in an 8-block CIFAR-10 model, yet these and a depth-scaled auxiliary term move accuracy by under one percentage point among non-degenerate variants; on Tiny ImageNet, a harder cross-dataset check of the selected configuration, layer health likewise does not track accuracy. A theory-derived attenuation-compensated objective raises deepest-block separation $5.9\times$ at a consistent cost in the FF classifier's own accuracy that a probe trained on the frozen features recovers; in a smaller CIFAR-10 control, most of its separation gain comes from dropping the depth-scaled auxiliary term it replaces. Accuracy invariance is a property of the deployed sum-based scoring rule under redistribution (Proposition 3.2) and, empirically, of the structured trained readouts we test, not of what the network learns: retrained variants disagree on 13-26% of test predictions. Architecture and augmentation matter far more than the non-degenerate training rules studied. Cumulative free-riding is therefore a real and repairable pathology, but for the softplus and squared-hinge objectives, architectures, and datasets we study, it is not the dominant factor limiting accuracy.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑