arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2203.15290cs.RO
收藏

使用强化学习控制神经元连接机器人的设计策略

Design strategies for controlling neuron-connected robots using reinforcement learning

The University of Tokyo(东京大学);Microsoft(微软)

Haruto Sawada, Naoki Wake, Kazuhiro Sasabuchi, Jun Takamatsu, Hirokazu Takahashi, Katsushi Ikeuchi

摘要:

尽管利用生物神经元的计算对机器人控制的兴趣越来越大,但神经元连接的机器人的上下文相关行为仍然是一个挑战。情境依赖行为在这里被定义为不是简单的感觉-运动耦合的结果,而是基于对任务目标的理解的行为。本文提出了基于任务目标训练神经元连接机器人的设计原则,以实现上下文相关的行为。首先,我们采用深度强化学习(RL)来实现考虑目标成就的训练。其次,我们提出了一个神经元模拟器作为基于记录的神经数据的概率分布,旨在表示生理上有效的神经动力学,同时避免复杂的建模与高计算成本。此外,我们建议在训练过程中更新模拟器,以弥合模拟和真实设置之间的差距。实验表明,机器人逐渐学习上下文相关的行为在杆平衡和机器人导航任务。此外,学习到的策略对于基于新神经数据的神经模拟器是有效的,并且通过在训练期间更新模拟器来提高任务性能。这些结果表明,所提出的设计原则的上下文相关的行为的神经元连接的机器人的有效性。

英文摘要:

Despite the growing interest in robot control utilizing the computation of biological neurons, context-dependent behavior by neuron-connected robots remains a challenge. Context-dependent behavior here is defined as behavior that is not the result of a simple sensory-motor coupling, but rather based on an understanding of the task goal. This paper proposes design principles for training neuron-connected robots based on task goals to achieve context-dependent behavior. First, we employ deep reinforcement learning (RL) to enable training that accounts for goal achievements. Second, we propose a neuron simulator as a probability distribution based on recorded neural data, aiming to represent physiologically valid neural dynamics while avoiding complex modeling with high computational costs. Furthermore, we propose to update the simulators during the training to bridge the gap between the simulation and the real settings. The experiments showed that the robot gradually learned context-dependent behaviors in pole balancing and robot navigation tasks. Moreover, the learned policies were valid for neural simulators based on novel neural data, and the task performance increased by updating the simulators during training. These results suggest the effectiveness of the proposed design principle for the context-dependent behavior of neuron-connected robots.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑