IF2Net：用于持续学习的天生无遗忘网络
IF2Net: Innately Forgetting-Free Networks for Continual Learning
AI总结：
本文提出IF2Net，一种通过冻结重用权重和正交投影实现持续学习无遗忘的框架，无需任务身份即可增量学习，兼顾稳定性与可塑性。
摘要：
持续学习能够在不干扰先前所学知识的情况下，增量式地吸收新概念。受神经网络特性（信息存储在连接权重中）的启发，我们研究了如何为持续学习场景设计一种天生无遗忘网络（IF2Net）。本研究提出了一种简单而有效的学习范式，其巧妙之处在于，在学习新任务前后，保持与每个已见任务相关的权重不变。我们首先提出了一种在任务序列上使用随机权重的新型表示级学习方法。该技术是指将由随机化引起的漂移表示调整回其各自的任务最优工作状态，但所涉及的权重被冻结并重用（与众所周知的逐层权重更新相反）。然后，通过将输出权重更新投影到简约的正交空间中，可以实现无遗忘的顺序决策，使得适应过程不干扰旧知识，同时保持模型的可塑性。IF2Net通过整合随机化和正交化各自的优势，使得单一网络能够在不于测试时告知任务身份的情况下，内在地学习无限映射规则。我们在广泛的理论分析和实证研究中验证了我们方法的有效性。
英文摘要：
Continual learning can incrementally absorb new concepts without interfering with previously learned knowledge. Motivated by the characteristics of neural networks, in which information is stored in weights on connections, we investigated how to design an Innately Forgetting-Free Network (IF2Net) for continual learning context. This study proposed a straightforward yet effective learning paradigm by ingeniously keeping the weights relative to each seen task untouched before and after learning a new task. We first presented the novel representation-level learning on task sequences with random weights. This technique refers to tweaking the drifted representations caused by randomization back to their separate task-optimal working states, but the involved weights are frozen and reused (opposite to well-known layer-wise updates of weights). Then, sequential decision-making without forgetting can be achieved by projecting the output weight updates into the parsimonious orthogonal space, making the adaptations not disturb old knowledge while maintaining model plasticity. IF2Net allows a single network to inherently learn unlimited mapping rules without telling task identities at test time by integrating the respective strengths of randomization and orthogonalization. We validated the effectiveness of our approach in the extensive theoretical analysis and empirical study.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
