Geo-LoRA：面向持续学习中低秩适应的几何感知子空间演化

中文摘要 AI 辅助

基于LoRA适配器的无重放类增量学习（CIL）仍具挑战性，因为跨任务更新的低秩子空间缺乏几何控制，导致共享表征不稳定且任务特定更新反复坍缩至先前占用的方向。本文提出Geo-LoRA，这是一个几何感知框架，可显式调控持续学习过程中共享和任务特定低秩子空间的演化。对于共享分支，子空间投影保留（SPP）约束连续更新遵循格拉斯曼流形上的平滑轨迹，自适应核心-松弛对齐（ACSA）将过渡分解为主成分和残差成分，对齐前者同时调制后者以平衡稳定性与可塑性。对于任务特定分支，中位数校准块重叠（MCBO）通过归一化投影重叠施加统计约束，惩罚过度重用以缓解子空间拥挤。这些约束共同调控各层及各任务的所有LoRA子空间的演化，且未引入超出标准LoRA的额外适配器类型。Geo-LoRA为持续低秩适应提供了原则性几何公式，在多个基准数据集和不同任务长度上始终达到最优性能。

英文摘要

Rehearsal-free class-incremental learning (CIL) with LoRA adapters remains challenging because the low-rank subspaces updated across tasks evolve without geometric control, causing unstable shared representations and repetitive collapse of task-specific updates into previously occupied directions. We introduce Geo-LoRA, a geometry-aware framework that explicitly regulates how low-rank subspaces, both shared and task-specific, evolve during continual learning. For the shared branch, Subspace Projection Preservation (SPP) constrains consecutive updates to follow smooth trajectories on the Grassmann manifold, and Adaptive Core-Slack Alignment (ACSA) decomposes transitions into principal and residual components, aligning the former while modulating the latter to balance stability and plasticity. For the task-specific branch, Median-Calibrated Block Overlap (MCBO) imposes a statistical constraint via normalized projection overlap, penalizing excessive reuse to mitigate subspace crowding. These constraints jointly regulate the evolution of all LoRA subspaces across layers and tasks without introducing additional adapter types beyond standard LoRA. Geo-LoRA provides a principled geometric formulation for continual low-rank adaptation and consistently achieves state-of-the-art performance across multiple benchmark datasets and different task lengths.

发表机构

• University of Electronic Science and Technology of China（电子科技大学）

机构由 AI 辅助整理，请以论文原文为准。