The Forward-Forward (FF) algorithm offers a biologically plausible alternative to backpropagation, enabling neural networks to learn through local updates. However, FF's efficacy relies heavily on the definition of "goodness", which is a scalar measure of neural activity. While current implementations predominantly utilize a simple sum-of-squares metric, it remains unclear if this default choice is optimal. To address this, we benchmarked 21 distinct goodness functions across four standard image datasets (MNIST, FashionMNIST, CIFAR-10, STL-10), evaluating classification accuracy, energy consumption, and carbon footprint. We found that certain alternative goodness functions inspired from various domains significantly outperform the standard baseline. Specifically, \texttt{game\_theoretic\_local} achieved 97.15\% accuracy on MNIST, \texttt{softmax\_energy\_margin\_local} reached 82.84\% on FashionMNIST, and \texttt{triplet\_margin\_local} attained 37.69\% on STL-10. Furthermore, we observed substantial variability in computational efficiency, highlighting a critical trade-off between predictive performance and environmental cost. These findings demonstrate that the goodness function is a pivotal hyperparameter in FF design. We release our code on \href{https://github.com/aryashah2k/In-Search-of-Goodness}{Github} for reference and reproducibility.
Deep Learning for ECG Analysis: Benchmarks and Insights from PTB-XL
Fraunhofer Heinrich Hertz Institute(弗劳恩霍夫海因里希·赫兹研究所);Physikalisch-Technische Bundesanstalt(联邦物理技术研究院);Technical University Berlin(柏林工业大学);King’s College London(伦敦国王学院)
GraIP: A Benchmarking Framework For Neural Graph Inverse Problems
Mila -- Quebec AI Institute, Montréal, Canada(魁北克AI研究所);Department of Computer Science, University of Stuttgart, Germany(斯图加特大学计算机科学系);International Max Planck Research School for Intelligent Systems, German(智能系统国际马克斯·普朗克研究学校);ETAS Research, Germany(ETAS研究);Faculty of Computer Science, RWTH Aachen University, Germany(亚琛工业大学计算机科学系);NEC Laboratories Europe(NEC欧洲实验室)
UAV3DCrop: Benchmarking 3D Reconstruction in Repeated Multi-Angle UAV Crop Surveys
University of Minnesota, Twin Cities(明尼苏达大学双城校区);University of Wisconsin–Madison(威斯康星大学麦迪逊分校);University of Pittsburgh(匹兹堡大学);Max Planck Institute for Biogeochemistry(马克斯·普朗克生物地球化学研究所);Boston University(波士顿大学);Peking University(北京大学)
MS MARCO: Benchmarking Ranking Models in the Large-Data Regime
Microsoft(微软);University College London(伦敦大学学院);University of Illinois, UC(伊利诺伊大学厄巴纳-香槟分校);University of Waterloo(滑铁卢大学)
TREC,CLEF,NTCIR和FIRE等评估工作,以及MS MARCO等公共排行榜,旨在鼓励研究和跟踪我们的进展,解决我们领域的重大问题。然而,目标不仅仅是确定哪个运行是“最好的”,获得最高分数。我们的目标是通过开发新的强大的技术来推动该领域的发展,这些技术在许多不同的环境中工作,并在研究和实践中采用。本文使用MS MARCO和TREC深度学习Track作为我们的案例研究,将其与20世纪90年代的TREC ad hoc排名案例进行比较。我们展示了如何设计的评价工作可以鼓励或阻止某些结果,并提出了内部和外部的有效性的结果的问题。我们对某些陷阱进行了一些分析,并陈述了避免此类陷阱的最佳实践。我们总结了到目前为止的工作进展,并描述了我们期望的“强大的有用性”的最终状态,以及可能需要采取的步骤。
Stochastic Forward-Forward Learning through Representational Dimensionality Compression
Institute of Science and Technology for Brain-Inspired Intelligence, Fudan University, Shanghai 200433, China(脑启发式智能科学与技术研究院,复旦大学,上海200433,中国);Key Laboratory of Computational Neuroscience and Brain-Inspired Intelligence (Fudan University), Ministry of Education, China(计算神经科学与脑启发式智能重点实验室(复旦大学),教育部,中国);MOE Frontiers Center for Brain Science, Fudan University, Shanghai 200433, China(教育部前沿脑科学中心(复旦大学),上海200433,中国);Ji Hua Laboratory, Foshan 528200, China(济华实验室,佛山528200,中国)
OpenBreastUS: Benchmarking Neural Operators for Wave Imaging Using Breast Ultrasound Computed Tomography
Peking University(北京大学);King Abdullah University of Science and Technology(阿卜杜拉国王科技大学);Institute of Acoustics, Chinese Academy of Sciences(中国科学院声学研究所);Yanqi Lake Beijing Institute of Mathematical Sciences and Applications(雁栖湖北京数学科学与应用研究院);DP Technology(深势科技);National Biomedical Imaging Center, Peking University(北京大学国家生物医学成像中心)