arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1709.06011cs.MAcs.AIcs.LGcs.SYeess.SYstat.ML
收藏

引导式深度强化学习用于群体系统

Guided Deep Reinforcement Learning for Swarm Systems

TU Darmstadt, Darmstadt, Germany(图宾根大学);University of Lincoln(林肯大学)

Maximilian Hüttenrauch, Adrian Šošić, Gerhard Neumann

AI总结:

本文研究如何通过有限感知能力的协作代理(如机器人群)学习控制方法,提出引导式强化学习框架,利用中央 critic 获取全局状态以简化策略评估,通过深度强化学习近似 Q 函数和策略。

摘要:

本文研究如何学习控制具有有限感知能力的协作代理群体(如机器人群)。代理仅具备基本传感器能力,但通过协作可完成复杂任务,如分布式装配或搜索救援。学习群体代理的策略因分布式部分可观测性而困难。本文采用引导式方法,其中 critic 在学习过程中拥有全局状态的中央访问,从而从强化学习角度简化策略评估问题。例如,通过摄像头图像获取所有机器人位置,但该图像仅供 critic 使用,不供机器人控制策略。本文采用 actor-critic 方法,其中 actor 仅基于本地感知信息做决策,而 critic 基于真实全局状态进行学习。算法使用深度强化学习近似 Q 函数和策略。算法性能在两个简单模拟 2D 代理任务上进行评估:1) 找到并维持一定距离;2) 定位目标。

英文摘要:

In this paper, we investigate how to learn to control a group of cooperative agents with limited sensing capabilities such as robot swarms. The agents have only very basic sensor capabilities, yet in a group they can accomplish sophisticated tasks, such as distributed assembly or search and rescue tasks. Learning a policy for a group of agents is difficult due to distributed partial observability of the state. Here, we follow a guided approach where a critic has central access to the global state during learning, which simplifies the policy evaluation problem from a reinforcement learning point of view. For example, we can get the positions of all robots of the swarm using a camera image of a scene. This camera image is only available to the critic and not to the control policies of the robots. We follow an actor-critic approach, where the actors base their decisions only on locally sensed information. In contrast, the critic is learned based on the true global state. Our algorithm uses deep reinforcement learning to approximate both the Q-function and the policy. The performance of the algorithm is evaluated on two tasks with simple simulated 2D agents: 1) finding and maintaining a certain distance to each others and 2) locating a target.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑