释放非中心化设备上持续学习的潜力:综述
Unleashing the Power of Continual Learning on Non-Centralized Devices: A Survey
AI总结:
本文综述了非中心化持续学习(NCCL)在分布式设备上处理非平稳流数据的研究进展,系统梳理了应对灾难性遗忘、分布偏移、异构性和隐私问题的现有方法,并建立大规模基准分析最新方法性能,指出未来研究方向。
摘要:
非中心化持续学习(NCCL)已成为一种新兴范式,使车辆和服务器等分布式设备能够处理来自联合非平稳环境的流数据。为了在分布式系统中部署该范式时实现高可靠性和可扩展性,必须克服来自空间和时间维度的挑战,这些挑战表现为分布偏移、灾难性遗忘、异构性和隐私问题。本综述聚焦于全面考察非中心化持续学习算法的发展及其在分布式设备上的实际部署。我们首先介绍非中心化学习和持续学习的背景与基础知识。然后,我们从三个层面回顾现有解决方案,以展示现有技术如何缓解灾难性遗忘和分布偏移。此外,我们深入探讨了各种类型的异构性问题、安全性和隐私属性,以及三种常见场景下的实际应用。进一步地,我们建立了一个大规模基准来重新审视该问题,并分析了最先进的NCCL方法的性能。最后,我们讨论了NCCL中的重要挑战和未来研究方向。
英文摘要:
Non-Centralized Continual Learning (NCCL) has become an emerging paradigm for enabling distributed devices such as vehicles and servers to handle streaming data from a joint non-stationary environment. To achieve high reliability and scalability in deploying this paradigm in distributed systems, it is essential to conquer challenges stemming from both spatial and temporal dimensions, manifesting as distribution shifts, catastrophic forgetting, heterogeneity, and privacy issues. This survey focuses on a comprehensive examination of the development of the non-centralized continual learning algorithms and the real-world deployment across distributed devices. We begin with an introduction to the background and fundamentals of non-centralized learning and continual learning. Then, we review existing solutions from three levels to represent how existing techniques alleviate the catastrophic forgetting and distribution shift. Additionally, we delve into the various types of heterogeneity issues, security, and privacy attributes, as well as real-world applications across three prevalent scenarios. Furthermore, we establish a large-scale benchmark to revisit this problem and analyze the performance of the state-of-the-art NCCL approaches. Finally, we discuss the important challenges and future research directions in NCCL.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。