Incorporating the Forward Forward algorithm into neural network training represents a transformative shift from traditional methods, introducing a dual forward mechanism that streamlines the learning process by bypassing the complexities of derivative propagation. This method is noted for its simplicity and efficiency and involves executing two forward passes the first with actual data to promote positive reinforcement, and the second with synthetically generated negative data to enable discriminative learning. Our experiments confirm that the Forward Forward algorithm is not merely an experimental novelty but a viable training strategy that competes robustly with conventional multi layer perceptron (MLP) architectures. To overcome the limitations inherent in traditional saliency techniques, which predominantly rely on gradient based methods, we developed a bespoke saliency algorithm specifically tailored for the Forward Forward framework. This innovative algorithm enhances the intuitive understanding of feature importance and network decision-making, providing clear visualizations of the data features most influential in model predictions. By leveraging this specialized saliency method, we gain deeper insights into the internal workings of the model, significantly enhancing our interpretative capabilities beyond those offered by standard approaches. Our evaluations, utilizing the MNIST and Fashion MNIST datasets, demonstrate that our method performs comparably to traditional MLP-based models.
Boosting Factorization Machines via Saliency-Guided Mixup
University of Science and Technology of China(中国科学技术大学);University of Arizona(亚利桑那大学);Huawei Noah’s Ark Lab(华为诺亚方舟实验室);JD Explore Academy(京东探索研究院);University of Sydney(悉尼大学)
Distinguishing between Normal and Cancer Cells Using Autoencoder Node Saliency
Lawrence Livermore National Laboratory(劳伦斯利弗莫尔国家实验室)
基因表达谱已被广泛用于表征细胞对疾病的反应模式。随着数据变得可用,可扩展的学习工具包对于使用深度学习模型来处理大型数据集以模拟复杂的生物过程变得至关重要。我们提出了一个自动编码器来捕获从基因表达谱中恢复的非线性关系。自动编码器是一种使用人工神经网络的非线性降维技术,它学习未标记数据的隐藏表示。我们训练的自动编码器上收集的肿瘤样本从国家癌症研究所基因组数据共享,并获得一个广义和无监督的潜在表示。我们利用专注于HPC的深度学习工具包Livermore Big Artificial神经网络(LBANN)来高效地并行化训练算法,将计算时间从几个小时减少到几分钟。使用经过训练的自动编码器,我们生成了一个小数据集的潜在表示,其中包含各种肿瘤类型的正常细胞和癌细胞对。一种新的措施,称为自动编码器节点显着性(ANS)的介绍,以确定隐藏的节点,最好区分各种对细胞。我们比较我们的最佳分类节点的主成分分析和可视化的t分布随机邻居嵌入的结果。我们证明了自动编码器可以有效地为数据集中的多个学习任务提取不同的基因特征。
On the explainable properties of 1-Lipschitz Neural Networks: An Optimal Transport Perspective
Université Paul-Sabatier(保罗萨巴蒂尔大学);Institut de Recherche Technologique Saint-Exupéry(圣埃克苏佩里技术研究院);Carney Institute for Brain Science,Brown University(布朗大学卡尼脑科学研究所);Innovation & Research Division, SNCF(法国国家铁路公司创新与研究部)