远端干扰：探索基于模型的持续学习的极限
Distal Interference: Exploring the Limits of Model-Based Continual Learning
AI总结：
本研究分析了持续学习中梯度下降导致的远端干扰问题，证明无此干扰的模型须指数级规模，并提出新型 ABEL-Spline 架构，但实验表明其远端干扰保证不足以支持仅依靠模型的持续学习。
摘要：
持续学习是机器学习模型对不同任务的顺序学习。持续学习已知会受到灾难性干扰或灾难性遗忘的阻碍，即在学习新任务时快速遗忘先前学过的任务。尽管人工神经网络（ANN）在实践中取得了成功，但它们容易受到灾难性干扰的影响。本研究分析了梯度下降以及远端输入点之间的重叠表示如何导致远端干扰和灾难性干扰。远端干扰是指在域的子集上训练模型会导致该域其他子集发生非局部变化的现象。本研究表明，没有远端干扰的均匀可训练模型必须具有指数级规模。提出了一种名为 ABEL-Spline 的新型反对称有界指数层 B 样条 ANN 架构，该架构可以逼近任何连续函数，是均匀可训练的，具有多项式计算复杂度，并为远端干扰提供了一些保证。实验展示了 ABEL-Spline 的理论特性。ABEL-Spline 还在基准回归问题上进行了评估。结论是，ABEL-Spline 中较弱的远端干扰保证不足以支持仅依靠模型的持续学习。据推测，使用多项式复杂度模型进行持续学习需要增强训练数据或算法。
英文摘要：
Continual learning is the sequential learning of different tasks by a machine learning model. Continual learning is known to be hindered by catastrophic interference or forgetting, i.e. rapid unlearning of earlier learned tasks when new tasks are learned. Despite their practical success, artificial neural networks (ANNs) are prone to catastrophic interference. This study analyses how gradient descent and overlapping representations between distant input points lead to distal interference and catastrophic interference. Distal interference refers to the phenomenon where training a model on a subset of the domain leads to non-local changes on other subsets of the domain. This study shows that uniformly trainable models without distal interference must be exponentially large. A novel antisymmetric bounded exponential layer B-spline ANN architecture named ABEL-Spline is proposed that can approximate any continuous function, is uniformly trainable, has polynomial computational complexity, and provides some guarantees for distal interference. Experiments are presented to demonstrate the theoretical properties of ABEL-Splines. ABEL-Splines are also evaluated on benchmark regression problems. It is concluded that the weaker distal interference guarantees in ABEL-Splines are insufficient for model-only continual learning. It is conjectured that continual learning with polynomial complexity models requires augmentation of the training data or algorithm.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
