分布式持续学习
Distributed Continual Learning
AI总结：
研究分布式持续学习与联邦学习的交叉领域，提出数学框架并设计三种信息共享模式，通过实验证明模块化参数共享在降低通信成本的同时实现最优性能。
摘要：
本工作研究了持续学习与联邦学习的交叉领域，其中独立的智能体在其环境中面临独特的任务，并逐步开发和共享知识。我们引入了一个数学框架，捕捉了分布式持续学习的核心方面，包括智能体模型与统计异构性、持续分布偏移、网络拓扑以及通信约束。基于分布式持续学习相较于单智能体学习能提升个体智能体性能这一论点，我们确定了三种信息交换模式：数据实例、完整模型参数和模块化（部分）模型参数。我们为每种共享模式开发了算法，并在各种数据集、拓扑结构和通信限制下进行了广泛的实证研究。我们的发现揭示了三个关键见解：随着任务变得更加复杂，共享参数比共享数据更高效；模块化参数共享在最小化通信成本的同时产生最佳性能；结合多种共享模式能累积提升性能。
英文摘要：
This work studies the intersection of continual and federated learning, in which independent agents face unique tasks in their environments and incrementally develop and share knowledge. We introduce a mathematical framework capturing the essential aspects of distributed continual learning, including agent model and statistical heterogeneity, continual distribution shift, network topology, and communication constraints. Operating on the thesis that distributed continual learning enhances individual agent performance over single-agent learning, we identify three modes of information exchange: data instances, full model parameters, and modular (partial) model parameters. We develop algorithms for each sharing mode and conduct extensive empirical investigations across various datasets, topology structures, and communication limits. Our findings reveal three key insights: sharing parameters is more efficient than sharing data as tasks become more complex; modular parameter sharing yields the best performance while minimizing communication costs; and combining sharing modes can cumulatively improve performance.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
