PLATE: 可塑性可调的几何感知持续学习高效适配器
PLATE: Plasticity-Tunable Efficient Adapters for Geometry-Aware Continual Learning
AI总结：
提出无需旧任务数据的持续学习方法PLATE，利用预训练网络的几何冗余性，通过结构化低秩更新显式控制可塑性-保留权衡，提升最坏情况保留保证。
摘要：
我们为预训练模型开发了一种持续学习方法，该方法不需要访问旧任务数据，解决了基础模型适应中预训练分布通常不可用的实际障碍。我们的关键观察是，预训练网络表现出大量的几何冗余性，并且这种冗余性可以通过两种互补的方式加以利用。首先，冗余神经元提供了预训练时代主导特征方向的代理，使得可以直接从预训练权重构建近似受保护的更新子空间。其次，冗余性为可塑性的放置位置提供了自然偏差：通过将更新限制在冗余神经元的子集并约束剩余的自由度，我们获得了在旧数据分布上功能漂移减少且最坏情况保留保证改善的更新族。这些见解导致了PLATE（可塑性可调的高效适配器），一种不需要过去任务数据的持续学习方法，它提供了对可塑性-保留权衡的显式控制。PLATE通过结构化低秩更新ΔW = B A Q^T参数化每一层，其中B和Q从预训练权重一次性计算并保持冻结，只有A在新任务上训练。代码可在https://this URL获取。
英文摘要：
We develop a continual learning method for pretrained models that \emph{requires no access to old-task data}, addressing a practical barrier in foundation model adaptation where pretraining distributions are often unavailable. Our key observation is that pretrained networks exhibit substantial \emph{geometric redundancy}, and that this redundancy can be exploited in two complementary ways. First, redundant neurons provide a proxy for dominant pretraining-era feature directions, enabling the construction of approximately protected update subspaces directly from pretrained weights. Second, redundancy offers a natural bias for \emph{where} to place plasticity: by restricting updates to a subset of redundant neurons and constraining the remaining degrees of freedom, we obtain update families with reduced functional drift on the old-data distribution and improved worst-case retention guarantees. These insights lead to \textsc{PLATE} (\textbf{Pla}sticity-\textbf{T}unable \textbf{E}fficient Adapters), a continual learning method requiring no past-task data that provides explicit control over the plasticity-retention trade-off. PLATE parameterizes each layer with a structured low-rank update $ΔW = B A Q^\top$, where $B$ and $Q$ are computed once from pretrained weights and kept frozen, and only $A$ is trained on the new task. The code is available at https://github.com/SalesforceAIResearch/PLATE.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
