持续学习领域不变表示
Continual Learning of Domain-Invariant Representations
AI总结：
本文提出持续学习领域不变表示方法，通过结合回放训练和定制的序列不变性对齐，提升模型在未见目标领域上的泛化能力。
摘要：
持续学习（CL）旨在在多个领域上依次训练模型，而不遗忘之前学习的知识。然而，现有CL方法优化的是领域内性能，因此容易学习到虚假的、领域特定的提示（"捷径学习"），这限制了部署后对未见领域的泛化能力。本文通过持续学习领域不变表示来解决这一限制。我们引入了一类广泛的方法，依次学习捕捉跨领域的不变结构的表示。我们的方法受到观察的启发，即这些不变结构往往保留了底层因果机制，这可以减少对领域特定提示的过拟合风险，从而提供更好的领域外泛化能力。我们提出的方法结合了基于回放的训练和定制的序列不变性对齐，以学习并保留时间上的不变结构。我们在面向部署的协议下评估了我们的方法，该协议测量在未见目标领域上的性能。在六个基准和现实世界的数据集上，涵盖视觉、医学、制造和生态领域，我们的方法在泛化到未见目标领域方面始终优于现有CL基线。作为消融研究，我们进一步表明，简单的顺序训练扩展现有领域不变表示学习（DIRL）方法只能提供有限的收益。据我们所知，这是首次开发用于持续学习的领域不变表示方法。
英文摘要：
Continual learning (CL) aims to train models sequentially over multiple domains without forgetting previously learned knowledge. However, existing CL methods optimize for in-domain performance and are therefore prone to learning spurious, domain-specific cues (``shortcut learning''), which limits generalization to unseen domains after deployment. In this paper, we address this limitation through continual learning of domain-invariant representation. We introduce a broad class of CL methods that sequentially learn representations capturing invariant structures across domains. Our methods are motivated by the observation that such invariant structures often preserve the underlying causal mechanisms, which can reduce the risk of overfitting to domain-specific cues and thus offer better out-of-domain generalization. Our proposed CL methods combine replay-based training with a tailored sequential invariance alignment to learn -- and preserve -- invariant structures over time. We evaluate our methods under a deployment-oriented protocol that measures performance on unseen target domains. Across six benchmark and real-world datasets spanning vision, medicine, manufacturing, and ecology, our methods consistently outperform existing CL baselines in terms of generalization to unseen target domains. As an ablation, we further show that naïve extensions of sequential training with existing domain-invariant representation learning (DIRL) methods provide only limited benefits. To the best of our knowledge, this is the first work to develop domain-invariant representation methods for CL.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
