arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2209.01682cs.NIcs.LG
收藏

基于无线信道的通信高效分布式学习

Communication Efficient Distributed Learning over Wireless Channels

Bar-Ilan University(巴伊兰大学)

Idan Achituve, Wenbo Wang, Ethan Fetaya, Amir Leshem

摘要:

垂直分布式学习利用多个学习工作者收集的局部特征来形成更好的全局模型。然而,工人和模型聚合器之间的数据交换参数训练招致了沉重的通信负担,特别是当学习系统是建立在容量受限的无线网络。在本文中,我们提出了一种新的分层分布式学习框架,其中每个工人分别学习他们的本地观测数据的低维嵌入。然后,它们执行通信高效的分布式最大池化,以有效地将合成的输入传输到聚合器。对于共享无线信道上的数据交换,我们提出了一个机会主义的载波侦听为基础的协议来实现最大池操作的输出数据从所有的学习工人。我们的模拟实验表明,所提出的学习框架是能够实现几乎相同的模型精度的学习模型使用的级联的所有原始输出的学习工人,同时需要一个通信负载是独立的工人的数量。

英文摘要:

Vertical distributed learning exploits the local features collected by multiple learning workers to form a better global model. However, the exchange of data between the workers and the model aggregator for parameter training incurs a heavy communication burden, especially when the learning system is built upon capacity-constrained wireless networks. In this paper, we propose a novel hierarchical distributed learning framework, where each worker separately learns a low-dimensional embedding of their local observed data. Then, they perform communication efficient distributed max-pooling for efficiently transmitting the synthesized input to the aggregator. For data exchange over a shared wireless channel, we propose an opportunistic carrier sensing-based protocol to implement the max-pooling operation for the output data from all the learning workers. Our simulation experiments show that the proposed learning framework is able to achieve almost the same model accuracy as the learning model using the concatenation of all the raw outputs from the learning workers, while requiring a communication load that is independent of the number of workers.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑