迈向联邦增量学习中的高效重放
Towards Efficient Replay in Federated Incremental Learning
AI总结：
针对联邦增量学习中数据异构和存储受限导致的灾难性遗忘问题，提出 Re-Fed 框架，通过协调客户端基于全局与局部重要性缓存样本进行重放训练，有效缓解遗忘并取得优异性能。
摘要：
在联邦学习（FL）中，每个客户端的数据通常被假定为固定或静态的。然而，在现实应用中，数据通常以增量方式到达，数据域可能会动态增加。在本研究中，我们研究了联邦增量学习（FIL）场景中数据异构性带来的灾难性遗忘问题，其中边缘客户端可能缺乏足够的存储空间来保留完整数据。我们提出采用一个简单、通用的 FIL 框架，名为 Re-Fed，它可以协调每个客户端缓存重要样本以进行重放。更具体地说，当新任务到达时，每个客户端首先根据样本的全局和局部重要性缓存选定的先前样本。然后，客户端使用缓存的样本和新任务中的样本来训练本地模型。在理论上，我们分析了 Re-Fed 发现重要样本进行重放的能力，从而缓解灾难性遗忘问题。此外，我们通过实验表明，与最先进的方法相比，Re-Fed 取得了具有竞争力的性能。
英文摘要：
In Federated Learning (FL), the data in each client is typically assumed fixed or static. However, data often comes in an incremental manner in real-world applications, where the data domain may increase dynamically. In this work, we study catastrophic forgetting with data heterogeneity in Federated Incremental Learning (FIL) scenarios where edge clients may lack enough storage space to retain full data. We propose to employ a simple, generic framework for FIL named Re-Fed, which can coordinate each client to cache important samples for replay. More specifically, when a new task arrives, each client first caches selected previous samples based on their global and local importance. Then, the client trains the local model with both the cached samples and the samples from the new task. Theoretically, we analyze the ability of Re-Fed to discover important samples for replay thus alleviating the catastrophic forgetting problem. Moreover, we empirically show that Re-Fed achieves competitive performance compared to state-of-the-art methods.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
