面向预训练模型的持续学习的任务锚定表示塑造
Task-Anchored Representation Shaping for Pre-Trained Model-Based Continual Learning
- Nanjing University（南京大学）
机构由 AI 辅助整理，请以论文原文为准。
AI总结：
该研究针对预训练模型持续学习中的跨任务歧义瓶颈，提出TAILS模块，通过任务锚点修正特征表示，以轻量开销提升分类与任务推理性能。
AI中文摘要：
预训练模型（PTMs）提供了稳定表示，为持续学习奠定了坚实基础，便于轻量适配新任务。然而，对每个任务适配良好并不能确保对所有已学习任务的可靠推理。由于任务边界通常是人为设定且语义纠缠的，即便拥有强大的PTM特征，来自未知任务的输入仍可能存在歧义，使得跨任务预测成为关键瓶颈。我们提出任务锚定推理潜在塑造（TAILS），这是一种轻量型后PTM模块，可集成到多种持续学习器中，并通过解耦步骤优化。TAILS使用固定任务锚点作为累积知识的持久参考，解释每个样本相对于这些参考的特征表示，再将跨任务的相关证据组合成潜在回忆。TAILS不选择特定任务路径或调整分类器输出，而是利用潜在回忆在预测前直接修正特征表示，因此在表示层面解决跨任务歧义，同时保留原始PTM、特定方法模块和分类器不变。在多个基于PTM的持续学习范式上进行的大量实验表明，TAILS可在适度参数开销和可忽略推理成本的情况下，提升分类和任务推理性能。
英文摘要：
Pre-trained models (PTMs) provide a strong foundation for continual learning by offering stable representations that facilitate lightweight adaptation to new tasks. However, adapting well to each task does not ensure reliable inference over all learned tasks. Since task boundaries are often artificial and semantically entangled, an input from an unknown task can remain ambiguous even with strong PTM features, making cross-task prediction a key bottleneck. We propose Task-Anchored Inference Latent Shaping (TAILS), a lightweight post-PTM module that can be integrated into diverse continual learners and optimized through a decoupled step. TAILS uses fixed task anchors as persistent references to accumulated knowledge. It interprets each sample's feature representation relative to these references, then composes relevant evidence across tasks into latent recall. Rather than selecting a task-specific path or adjusting classifier outputs, TAILS uses latent recall to directly correct the feature representation before prediction. It therefore resolves cross-task ambiguity at the representation level, while leaving the original PTM, method-specific modules, and classifier unchanged. Extensive experiments across multiple PTM-based continual learning paradigms show that TAILS can improve classification and task-inference performance with modest parameter overhead and negligible inference cost.
