用于持续学习任务的对比对称前向-前向算法（SFFA）
A Contrastive Symmetric Forward-Forward Algorithm (SFFA) for Continual Learning Tasks
AI总结：
本文提出对称前向-前向算法SFFA，通过正负神经元划分与对称局部损失改善FFA梯度不平衡，并验证其在图像分类和持续学习中的性能。
摘要：
所谓的前向-前向算法（Forward-Forward Algorithm，FFA）近来作为神经网络学习中传统反向传播算法的替代方案而受到关注，并在多种建模任务上取得了具有竞争力的性能。通过用两次对比性前向传播替代梯度反向传播的反向传播过程，FFA能够采用逐层训练启发式方法，从而避免其前身所经历的若干缺陷（例如梯度消失/梯度爆炸）。在分类任务中，这种对比方法已被证明能够有效地为输入数据创建潜在稀疏表示，最终有利于可区分性。然而，由于正数据与负数据之间的损失函数不平衡，FFA表现出固有的非对称梯度行为，对模型的泛化能力产生不利影响，并导致准确率下降。为解决这一问题，本工作提出了对称前向-前向算法（Symmetric Forward-Forward Algorithm，SFFA），这是对原始FFA的一种新颖修改，将每一层划分为正神经元和负神经元。这使得局部适应度函数可被定义为正神经元激活与整层活动之间的比率，从而在训练阶段形成对称的损失景观。为评估本方法增强的收敛性，我们使用多个图像分类基准进行了若干实验，将采用SFFA训练的模型准确率与采用对应FFA训练的模型准确率进行比较。作为这种重新表述的副产品，我们探索了在持续学习（Continual Learning，CL）任务中使用逐层训练算法的优势。逐层训练算法所诱导的神经元专门化及其激活的稀疏性，使高效的CL策略能够将新知识（类别）整合到神经网络中，同时防止对先前……的灾难性遗忘。
英文摘要：
The so-called Forward-Forward Algorithm (FFA) has recently gained momentum as an alternative to the conventional back-propagation algorithm for neural network learning, yielding competitive performance across various modeling tasks. By replacing the backward pass of gradient back-propagation with two contrastive forward passes, the FFA avoids several shortcomings undergone by its predecessor (e.g., vanishing/exploding gradient) by enabling layer-wise training heuristics. In classification tasks, this contrastive method has been proven to effectively create a latent sparse representation of the input data, ultimately favoring discriminability. However, FFA exhibits an inherent asymmetric gradient behavior due to an imbalanced loss function between positive and negative data, adversely impacting on the model's generalization capabilities and leading to an accuracy degradation. To address this issue, this work proposes the Symmetric Forward-Forward Algorithm (SFFA), a novel modification of the original FFA which partitions each layer into positive and negative neurons. This allows the local fitness function to be defined as the ratio between the activation of positive neurons and the overall layer activity, resulting in a symmetric loss landscape during the training phase. To evaluate the enhanced convergence of our method, we conduct several experiments using multiple image classification benchmarks, comparing the accuracy of models trained with SFFA to those trained with its FFA counterpart. As a byproduct of this reformulation, we explore the advantages of using a layer-wise training algorithm for Continual Learning (CL) tasks. The specialization of neurons and the sparsity of their activations induced by layer-wise training algorithms enable efficient CL strategies that incorporate new knowledge (classes) into the neural network, while preventing catastrophic forgetting of previously...
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
