抗灾难性遗忘的一次式联邦学习
Catastrophic Forgetting Resilient One-Shot Incremental Federated Learning
AI总结：
OSI-FL通过一次通信轮次生成类别特定嵌入并结合选择性样本保留技术，有效解决增量联邦学习中的通信开销和灾难性遗忘问题。
摘要：
现代大数据系统生成大量、异构且地理分布广泛的流，这些数据规模大且敏感，使集中化处理变得困难。尽管联邦学习（FL）提供了一种增强隐私的训练机制，但其假设数据流是静态的，并在多个轮次中学习协作模型，这使得在通信受限的场景中学习增量数据具有挑战性。本文提出了一次式增量联邦学习（OSI-FL），这是首个解决通信开销和灾难性遗忘双重挑战的FL框架。OSI-FL在单次通信轮次中，由每个客户端使用冻结的视觉语言模型（VLM）生成类别特定的嵌入，并由服务器上预训练的扩散模型用来合成与客户端数据分布相似的新数据样本。这些合成样本用于服务器端训练。然而，仍然存在两个挑战：i）逐渐到来的任务需要重新训练全局模型；ii）随着未来任务的到来，重新训练模型会引入灾难性遗忘。为此，我们通过选择性样本保留（SSR）来增强训练，该方法根据样本损失识别并保留每个类别和任务对的前p个最有信息量的样本。SSR通过确保代表性保留的样本在后续迭代中被纳入训练来限制遗忘。实验结果表明，OSI-FL在三个基准数据集的类别增量和领域增量场景中均优于传统和一次式FL方法。
英文摘要：
Modern big-data systems generate massive, heterogeneous, and geographically dispersed streams that are large-scale and privacy-sensitive, making centralization challenging. While federated learning (FL) provides a privacy-enhancing training mechanism, it assumes a static data flow and learns a collaborative model over multiple rounds, making learning with \textit{incremental} data challenging in limited-communication scenarios. This paper presents One-Shot Incremental Federated Learning (OSI-FL), the first FL framework that addresses the dual challenges of communication overhead and catastrophic forgetting. OSI-FL communicates category-specific embeddings, devised by a frozen vision-language model (VLM) from each client in a single communication round, which a pre-trained diffusion model at the server uses to synthesize new data similar to the client's data distribution. The synthesized samples are used on the server for training. However, two challenges still persist: i) tasks arriving incrementally need to retrain the global model, and ii) as future tasks arrive, retraining the model introduces catastrophic forgetting. To this end, we augment training with Selective Sample Retention (SSR), which identifies and retains the top-p most informative samples per category and task pair based on sample loss. SSR bounds forgetting by ensuring that representative retained samples are incorporated into training in further iterations. The experimental results indicate that OSI-FL outperforms baselines, including traditional and one-shot FL approaches, in both class-incremental and domain-incremental scenarios across three benchmark datasets.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
