MAcPNN:具有时间依赖性的数据流上的相互协助学习
MAcPNN: Mutual Assisted Learning on Data Streams with Temporal Dependence
AI总结:
MAcPNN通过自主设备间相互协助学习,有效应对数据流中的概念漂移和时间依赖性,提升性能。
摘要:
物联网(IoT)分析通常涉及在数据流上应用机器学习(ML)模型。在这样的场景中,传统ML范式在处理概念漂移、时间依赖性和避免遗忘方面面临障碍。此外,在IoT中,不同边缘设备构建起一个网络。当在这些设备上学习模型时,连接它们可能有助于提高性能并重用其他设备的知识。本文提出了相互协助学习,这是一种基于维果茨基流行的社会文化认知发展理论的学习范式。每个设备都是自主的,不需要中央协调器。每当由于概念漂移导致性能下降时,它会向他人请求帮助,并决定他人的知识是否对解决新问题有用。这样,连接数量比经典联邦学习方法大幅减少,后者在每次训练轮次中都要求设备通信。每个设备都配备了连续渐进神经网络(cPNN)以处理数据流的动态特性。我们称这种实现为相互协助cPNN(MAcPNN)。为了实现它,我们允许cPNN对单个数据点进行预测,并应用量化以减少内存占用。实验结果证明了MAcPNN在提升合成和真实数据流性能方面的有效性。
英文摘要:
Internet of Things (IoT) Analytics often involves applying machine learning (ML) models on data streams. In such scenarios, traditional ML paradigms face obstacles related to continuous learning while dealing with concept drifts, temporal dependence, and avoiding forgetting. Moreover, in IoT, different edge devices build up a network. When learning models on those devices, connecting them could be useful in improving performance and reusing others' knowledge. This work proposes Mutual Assisted Learning, a learning paradigm grounded on Vygotsky's popular Sociocultural Theory of Cognitive Development. Each device is autonomous and does not need a central orchestrator. Whenever it degrades its performance due to a concept drift, it asks for assistance from others and decides whether their knowledge is useful for solving the new problem. This way, the number of connections is drastically reduced compared to the classical Federated Learning approaches, where the devices communicate at each training round. Every device is equipped with a Continuous Progressive Neural Network (cPNN) to handle the dynamic nature of data streams. We call this implementation Mutual Assisted cPNN (MAcPNN). To implement it, we allow cPNNs for single data point predictions and apply quantization to reduce the memory footprint. Experimental results prove the effectiveness of MAcPNN in boosting performance on synthetic and real data streams.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。