arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2207.09372cs.ROcs.AI
收藏

多机器人场景下的分散式联邦强化学习研究

On Decentralizing Federated Reinforcement Learning in Multi-Robot Scenarios

Federal Institute of Science and Technology(联邦科学技术学院);Indian Institute of Technology Guwahati(印度理工学院古瓦哈提分校)

Jayprakash S. Nair, Divya D. Kulkarni, Ajitem Joshi, Sruthy Suresh

摘要:

联邦学习(FL)允许跨多个计算设备协作地聚合学习到的信息并在它们之间共享相同的信息,从而解决隐私问题和对巨大带宽的需求。FL技术通常使用中央服务器或云来聚合从设备接收的模型。这种集中式FL技术存在固有的问题,例如中心节点的故障和信道带宽的瓶颈。当FL与作为设备的连接机器人结合使用时,中央控制实体的故障可能导致混乱的情况。本文描述了一种基于移动代理的范例分散FL在多机器人的情况下。使用Webots,一个流行的免费开源机器人模拟器,和Tartarus,一个移动代理平台,我们提出了一种方法来分散联邦学习在一组连接的机器人。随着Webots运行在不同的连接计算系统,我们展示了移动代理如何执行分散式联邦强化学习(dFRL)的任务。从使用Q-学习和SARSA通过聚合其相应的Q表进行的实验中获得的结果显示了在机器人领域使用分散FL的可行性。由于所提出的工作可以与其他学习算法和真正的机器人一起使用,它可以作为一个重要的工具,分散FL同时使用异构学习算法在多机器人的情况下的研究。

英文摘要:

Federated Learning (FL) allows for collaboratively aggregating learned information across several computing devices and sharing the same amongst them, thereby tackling issues of privacy and the need of huge bandwidth. FL techniques generally use a central server or cloud for aggregating the models received from the devices. Such centralized FL techniques suffer from inherent problems such as failure of the central node and bottlenecks in channel bandwidth. When FL is used in conjunction with connected robots serving as devices, a failure of the central controlling entity can lead to a chaotic situation. This paper describes a mobile agent based paradigm to decentralize FL in multi-robot scenarios. Using Webots, a popular free open-source robot simulator, and Tartarus, a mobile agent platform, we present a methodology to decentralize federated learning in a set of connected robots. With Webots running on different connected computing systems, we show how mobile agents can perform the task of Decentralized Federated Reinforcement Learning (dFRL). Results obtained from experiments carried out using Q-learning and SARSA by aggregating their corresponding Q-tables, show the viability of using decentralized FL in the domain of robotics. Since the proposed work can be used in conjunction with other learning algorithms and also real robots, it can act as a vital tool for the study of decentralized FL using heterogeneous learning algorithms concurrently in multi-robot scenarios.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑