发表机构
Tianjin University（天津大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
针对持续学习中低秩适配的谱可塑性坍缩问题，MuLoRA通过历史白化与动量更新的近似极正交化联合控制容量分配与利用，在多个类增量基准上取得最优平均准确率。
AI 中文摘要
低秩适配（LoRA）为持续学习提供了一种参数高效的方法，但其名义秩可能掩盖有效适配能力的损失。我们识别出“谱可塑性坍缩”现象：在顺序适配过程中，更新能量集中于少量奇异模态，导致大量可用的低秩空间未被充分利用。这揭示了仅靠干扰规避的局限性：保护历史表征并不能确保剩余适配能力对新任务有响应或被有效利用。为解决此问题，我们提出MuLoRA，它联合控制容量分配与利用。首先，历史白化识别出相对于累积历史响应具有强当前任务响应的输入方向，从而产生在训练期间保持固定的任务自适应基。其次，对动量更新进行近似极正交化，以降低所选空间内的谱集中度。一个标准正交基通过将因子更新谱精确传递到诱导权重更新，连接了这些机制。我们建立了精确子空间选择的极大-极小刻画，并在受控跨步各向异性下推导了累积谱界。在五个类增量基准和八个增量设置中，MuLoRA在16项报告指标中的15项上取得了最高平均准确率。
英文摘要
Low-rank adaptation (LoRA) provides a parameter-efficient approach to continual learning, but its nominal rank can conceal a loss of effective adaptation capacity. We identify \emph{spectral plasticity collapse}: during sequential adaptation, update energy becomes concentrated in a small subset of singular modes, leaving much of the available low-rank space underutilized. This exposes a limitation of interference avoidance alone: protecting historical representations does not ensure that the remaining adaptation capacity is responsive to new tasks or effectively utilized. To address this problem, we propose \texttt{MuLoRA}, which jointly controls capacity allocation and utilization. First, historical whitening identifies input directions with strong current-task response relative to accumulated historical response, yielding a task-adaptive basis that remains fixed during training. Second, approximate polar orthogonalization of momentum updates reduces spectral concentration within theselected space. An orthonormal basis connects these mechanisms by transferring the factor-update spectrum exactly tothe induced weight update. We establish a max--min characterization of exact subspace selection and derive cumulative spectral bounds under controlled cross-step anisotropy. Across five class-incremental benchmarks and eight incremental settings, \texttt{MuLoRA} achieves the highest mean accuracy in 15 of 16 reported metrics.
