边缘无处不在的分布式深度强化学习：分析离散动作空间中的拜占庭代理
Ubiquitous Distributed Deep Reinforcement Learning at the Edge: Analyzing Byzantine Agents in Discrete Action Spaces
摘要：
边缘计算在下一代移动网络中的集成正在为无数的网络物理系统带来低延迟和高带宽的无处不在的连接。这将进一步推动嵌入各种类型的自治系统边缘的日益增长的智能，其中协作机器学习有可能发挥重要作用。本文讨论了多智能体分布式深度强化学习中的一些挑战，这些挑战可能会在存在拜占庭或故障智能体的情况下发生。随着模拟与现实之间的差距得到弥合，必须考虑故障或错误的可能性。我们展示了错误的离散行为如何显着影响协作学习努力。特别是，我们分析的影响，有一小部分的代理，可能会执行错误的行动与给定的概率。我们研究的能力，系统收敛到一个共同的工作政策，通过协作学习过程的基础上，从每个代理的经验，以聚合为每个政策更新的数量，以及从代理遇到故障的错误动作的分数。我们的实验是在一个模拟环境中进行的，使用Atari测试平台的离散动作空间，和优势演员评论家（A2C）的分布式多智能体训练。
英文摘要：
The integration of edge computing in next-generation mobile networks is bringing low-latency and high-bandwidth ubiquitous connectivity to a myriad of cyber-physical systems. This will further boost the increasing intelligence that is being embedded at the edge in various types of autonomous systems, where collaborative machine learning has the potential to play a significant role. This paper discusses some of the challenges in multi-agent distributed deep reinforcement learning that can occur in the presence of byzantine or malfunctioning agents. As the simulation-to-reality gap gets bridged, the probability of malfunctions or errors must be taken into account. We show how wrong discrete actions can significantly affect the collaborative learning effort. In particular, we analyze the effect of having a fraction of agents that might perform the wrong action with a given probability. We study the ability of the system to converge towards a common working policy through the collaborative learning process based on the number of experiences from each of the agents to be aggregated for each policy update, together with the fraction of wrong actions from agents experiencing malfunctions. Our experiments are carried out in a simulation environment using the Atari testbed for the discrete action spaces, and advantage actor-critic (A2C) for the distributed multi-agent training.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
