一种用于可解释神经网络与持续学习的基于NMF的构建模块
An NMF-Based Building Block for Interpretable Neural Networks With Continual Learning
AI总结：
本文提出一种基于NMF的预测因子化耦合（PFC）模块，通过结合监督神经网络训练实现高预测性能与可解释性的平衡，并在持续学习、非独立同分布数据训练等场景中展现优势。
摘要：
现有的学习方法通常难以平衡可解释性与预测性能。虽然最近邻和非负矩阵分解（NMF）等模型具有高度的可解释性，但它们在监督学习任务上的预测性能通常有限。相比之下，基于多层感知机（MLP）的神经网络支持模块化构建富有表达能力的架构，并且往往具有更好的识别准确率，但在可解释性方面通常被视为黑盒。我们的方法旨在通过使用基于NMF的构建模块在这两个方面之间取得更好的平衡，该模块结合了有监督的神经网络训练方法，以在实现高预测性能的同时保留NMF理想的可解释性特性。我们在小型数据集上评估了我们的预测因子化耦合（PFC）模块，并表明它实现了与MLP相媲美的预测性能，同时还提供了改进的可解释性。我们展示了该方法在各种场景下的优势，例如持续学习、在非独立同分布数据上进行训练以及训练后的知识移除。此外，我们展示了使用PFC模块构建更具表达能力的架构的示例，包括一个全连接残差网络以及一个因子化循环神经网络（RNN），该网络在与普通RNN竞争时表现出色，同时提供了更好的可解释性。PFC模块使用一种收敛到不动点的迭代推理算法，这使得在训练后能够权衡准确性与计算量，但目前也阻碍了它在某些场景中作为通用MLP替代品的使用，例如在超大型数据集上进行训练。我们在 https://github.com/bkvogel/pfc 提供了源代码。
英文摘要：
Existing learning methods often struggle to balance interpretability and predictive performance. While models like nearest neighbors and non-negative matrix factorization (NMF) offer high interpretability, their predictive performance on supervised learning tasks is often limited. In contrast, neural networks based on the multi-layer perceptron (MLP) support the modular construction of expressive architectures and tend to have better recognition accuracy but are often regarded as black boxes in terms of interpretability. Our approach aims to strike a better balance between these two aspects through the use of a building block based on NMF that incorporates supervised neural network training methods to achieve high predictive performance while retaining the desirable interpretability properties of NMF. We evaluate our Predictive Factorized Coupling (PFC) block on small datasets and show that it achieves competitive predictive performance with MLPs while also offering improved interpretability. We demonstrate the benefits of this approach in various scenarios, such as continual learning, training on non-i.i.d. data, and knowledge removal after training. Additionally, we show examples of using the PFC block to build more expressive architectures, including a fully-connected residual network as well as a factorized recurrent neural network (RNN) that performs competitively with vanilla RNNs while providing improved interpretability. The PFC block uses an iterative inference algorithm that converges to a fixed point, making it possible to trade off accuracy vs computation after training but also currently preventing its use as a general MLP replacement in some scenarios such as training on very large datasets. We provide source code at https://github.com/bkvogel/pfc
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
