arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2209.12277eess.SP
收藏

能量受限无线网络中知识辅助的联邦学习

Knowledge-aided Federated Learning for Energy-limited Wireless Networks

Zhixiong Chen, Wenqiang Yi, Yuanwei Liu, Arumugam Nallanathan

摘要:

传统的基于模型聚合的联邦学习(FL)方法要求所有局部模型具有相同的体系结构,无法支持具有异构局部模型的实际场景。此外,频繁的模型交换对于资源有限的无线网络来说是昂贵的,因为现代深度神经网络通常有超过一百万个参数。为了应对这些挑战,我们提出了一种新的知识辅助FL(KFL)框架,它在每轮学习过程中聚合轻的高级数据特征,即知识。该框架允许设备独立设计其机器学习模型,并减少训练过程中的通信开销。然后,我们从理论上分析了所提出的框架的收敛界,揭示了在每一轮调度更多的数据量有助于提高学习性能。此外,如果整个学习过程中的总调度数据量是固定的,则应在早期轮次中调度大数据量。受此启发,我们定义了一个新的目标函数,即,加权调度数据样本量,将不明确的全局损耗最小化问题转化为易于处理的问题,用于设备调度、带宽分配和功率控制。为了处理未知的时变无线信道,我们考虑的问题转化为一个确定性的问题,每一轮的协助下,李雅普诺夫优化框架。然后,我们推导出最优的带宽分配和功率控制解决方案,并开发了一个有效的在线设备调度算法,以实现在学习过程中的能量学习权衡。在MNIST和CIFAR-10上的实验结果表明,与传统的基于模型聚合的算法相比,该算法能够减少99%以上的通信开销,同时获得更好的学习性能.

英文摘要:

The conventional model aggregation-based federated learning (FL) approach requires all local models to have the same architecture, which fails to support practical scenarios with heterogeneous local models. Moreover, frequent model exchange is costly for resource-limited wireless networks since modern deep neural networks usually have over a million parameters. To tackle these challenges, we propose a novel knowledge-aided FL (KFL) framework, which aggregates light high-level data features, namely knowledge, in the per-round learning process. This framework allows devices to design their machine-learning models independently and reduces the communication overhead in the training process. We then theoretically analyze the convergence bound of the proposed framework, revealing that scheduling more data volume in each round helps to improve the learning performance. In addition, large data volume should be scheduled in early rounds if the total scheduled data volume during the entire learning course is fixed. Inspired by this, we define a new objective function, i.e., the weighted scheduled data sample volume, to transform the inexplicit global loss minimization problem into a tractable one for device scheduling, bandwidth allocation, and power control. To deal with unknown time-varying wireless channels, we transform the considered problem into a deterministic problem for each round with the assistance of the Lyapunov optimization framework. Then, we derive the optimal bandwidth allocation and power control solution and develop an efficient online device scheduling algorithm to achieve an energy-learning trade-off in the learning process. Experimental results on MNIST and CIFAR-10 show that the proposed KFL is capable of reducing over 99% communication overhead while achieving better learning performance than the conventional model aggregation-based algorithms.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑