arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2511.18567cs.LG
收藏

寻找良好性:用于前向-前向算法的大型基准测试良好函数

In Search of Goodness: Large Scale Benchmarking of Goodness Functions for the Forward-Forward Algorithm

Department of Computer Science, Indian Institute of Technology Gandhinagar(计算机科学系,印度理工学院冈丁加尔)

Arya Shah, Vaibhav Tripathi

AI总结:

本文通过对比不同良好函数在图像数据集上的表现,发现某些函数在准确率和能耗方面优于标准方法,揭示了良好函数在FF算法设计中的关键作用。

摘要:

前向-前向(FF)算法提供了一种生物合理性的替代方案,用于反向传播,使神经网络能够通过局部更新来学习。然而,FF的效果严重依赖于“良好性”的定义,这是一个衡量神经活动的标量指标。尽管当前实现主要使用简单的平方和度量,但这种默认选择是否最优仍不清楚。为此,我们对四个标准图像数据集(MNIST、FashionMNIST、CIFAR-10、STL-10)上的21种不同的良好函数进行了基准测试,评估分类准确率、能耗和碳足迹。我们发现,某些受不同领域启发的替代良好函数显著优于标准基线。具体来说,game_theoretic_local在MNIST上实现了97.15%的准确率,softmax_energy_margin_local在FashionMNIST上达到了82.残84%,triplet_margin_local在STL-10上达到了37.69%。此外,我们观察到计算效率存在显著差异,突显了预测性能与环境成本之间的关键权衡。这些发现表明,良好函数是FF设计中的关键超参数。我们将在Github上发布我们的代码供参考和复现。

英文摘要:

The Forward-Forward (FF) algorithm offers a biologically plausible alternative to backpropagation, enabling neural networks to learn through local updates. However, FF's efficacy relies heavily on the definition of "goodness", which is a scalar measure of neural activity. While current implementations predominantly utilize a simple sum-of-squares metric, it remains unclear if this default choice is optimal. To address this, we benchmarked 21 distinct goodness functions across four standard image datasets (MNIST, FashionMNIST, CIFAR-10, STL-10), evaluating classification accuracy, energy consumption, and carbon footprint. We found that certain alternative goodness functions inspired from various domains significantly outperform the standard baseline. Specifically, \texttt{game\_theoretic\_local} achieved 97.15\% accuracy on MNIST, \texttt{softmax\_energy\_margin\_local} reached 82.84\% on FashionMNIST, and \texttt{triplet\_margin\_local} attained 37.69\% on STL-10. Furthermore, we observed substantial variability in computational efficiency, highlighting a critical trade-off between predictive performance and environmental cost. These findings demonstrate that the goodness function is a pivotal hyperparameter in FF design. We release our code on \href{https://github.com/aryashah2k/In-Search-of-Goodness}{Github} for reference and reproducibility.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑