InTAct: 基于区间的方法任务激活整合用于持续学习
InTAct: Interval-based Task Activation Consolidation for Continual Learning
AI总结：
InTAct通过区间方法整合任务激活，提供函数不变性保证，提升持续学习性能。
摘要：
持续学习是人工智能中的基本挑战，要求网络在获取新知识的同时保持之前学习的表示。尽管各种方法取得了成功，但大多数现有范式并未提供严格的数学保证以防止灾难性遗忘。目前提供此类保证的方法主要集中在使用区间算术（IA）分析参数空间，如InterContiNet框架。然而，限制高维权重更新可能计算成本很高。在本工作中，我们提出了InTAct（基于区间的方法任务激活整合），通过在神经元层面强制函数不变性来缓解灾难性遗忘。我们识别出特定的激活区间，其中以前的任务存在，并在这些区域内限制更新，同时在其他地方允许灵活适应。通过确保在这些嵌套激活区间内预测保持稳定，我们提供了一种可处理的数学不变性保证。我们强调，调节激活空间比参数约束要高效得多，因为内部信号的维度远低于模型权重的广阔空间。尽管我们的方法是架构无关的，适用于各种持续学习设置，但其与基于提示的方法的结合使其在具有挑战性的基准上实现了最先进的性能。
英文摘要：
Continual learning is a fundamental challenge in artificial intelligence that requires networks to acquire new knowledge while preserving previously learned representations. Despite the success of various approaches, most existing paradigms do not provide rigorous mathematical guarantees against catastrophic forgetting. Current methods that offer such guarantees primarily focus on analyzing the parameter space using \textit{interval arithmetic (IA)}, as seen in frameworks such as InterContiNet. However, restricting high-dimensional weight updates can be computationally expensive. In this work, we propose InTAct (Interval-based Task Activation Consolidation), a method that mitigates catastrophic forgetting by enforcing functional invariance at the neuron level. We identify specific activation intervals where previous tasks reside and constrain updates within these regions while allowing for flexible adaptation elsewhere. By ensuring that predictions remain stable within these nested activation intervals, we provide a tractable mathematical guarantee of functional invariance. We emphasize that regulating the activation space is significantly more efficient than parameter-based constraints, because the dimensionality of internal signals is much lower than that of the vast space of model weights. While our approach is architecture-agnostic and applicable to various continual learning settings, its integration with prompt-based methods enables it to achieve state-of-the-art performance on challenging benchmarks.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
