ReSCENE：联邦持续学习中灾难性遗忘的结构性缓解的服务器端重放
ReSCENE: Server-Side Replay for Structural Mitigation of Catastrophic Forgetting in Federated Continual Learning
- Texas A&M University（德克萨斯A&M大学）
- Inha University（仁荷大学）
机构由 AI 辅助整理，请以论文原文为准。
AI总结：
ReSCENE通过客户端上传数据替代物并由服务器重放，结构性缓解联邦持续学习中的灾难性遗忘，在多个数据集上显著提升准确率并大幅降低通信与计算开销。
AI中文摘要：
联邦持续学习必须随时间整合新任务，同时不丢失早期任务的知识。现有大多数方法将防遗忘机制附加到联邦学习的客户端训练、服务器聚合循环中，这为了保留早期知识而抑制新学习，并给资源受限的客户端带来负担。我们提出ReSCENE，通过让每个客户端上传其本地数据的小型压缩替代物，而服务器保留过去任务的替代物，并将当前任务替代物与之一同训练全局模型，从而在结构上缓解灾难性遗忘。为高效利用服务器内存，我们引入时间牧群（temporal herding），从任务期间累积的替代物池中选择较新的替代物，压缩至一个缓冲区。我们的研究提供了理论分析，表明该缓冲区能比完全累积所有替代物更紧密地代表原始任务数据。在CIFAR-10、CIFAR-100和TinyImageNet上，ReSCENE在七个基线中实现了最强准确率，平均准确率最高提升31.1个百分点，同时仅需低至0.11倍的客户端计算量，且上传量比模型更新基线最多减少179倍。ReSCENE在扩展到更大客户端群体和更大模型时仍保持高效，进一步展示了其有效性，使其成为联邦持续学习中一种实用方法。
英文摘要：
Federated continual learning must integrate new tasks over time without losing earlier-task knowledge. Most existing methods attach an anti-forgetting mechanism to the client-trained, server-aggregated loop of federated learning, which holds back new learning to preserve earlier knowledge and burdens resource-constrained clients. We propose ReSCENE, which structurally mitigates catastrophic forgetting by having each client upload a small condensed surrogate of its local data while the server keeps the surrogates of past tasks and trains the global model on them together with the current task surrogates. For efficient server memory, we introduce temporal herding, which selects the more recent surrogates from the pool accumulated over a task into a compressed buffer. Our study provides a theoretical analysis showing that this buffer can represent the original task data more closely than full accumulation of all surrogates. Across CIFAR-10, CIFAR-100, and TinyImageNet, ReSCENE achieves the strongest accuracy over seven baselines, by up to $31.1$ points of average accuracy, while requiring as little as $0.11\times$ of the client computation and up to $179\times$ less upload than the model-update baselines. ReSCENE further demonstrates its effectiveness when scaled to larger client populations and larger models while remaining efficient, which makes it a practical method for federated continual learning.
