arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2106.08422cs.RO
收藏

基于深度模态混合网络的机器人模仿与镜像系统

Imitation and Mirror Systems in Robots through Deep Modality Blending Networks

Bogazici University(海峡大学);The University of Tokyo(东京大学);Osaka University(大阪大学);Ozyegin University(厄兹耶金大学)

M. Y. Seker, A. Ahmetoglu, Y. Nagai, M. Asada, E. Oztop, E. Ugur

摘要:

学习与环境的交互不仅使智能体具有操纵能力,而且还产生信息,以促进行为理解和模仿能力的建立。这似乎是生物系统,特别是灵长类动物所采用的策略,镜像神经元的存在证明了这一点,镜像神经元似乎参与了多模态动作理解。如何从机器人的交互经验中受益,以使其他代理的行为和目标的理解仍然是一个具有挑战性的问题。在这项研究中,我们提出了一种新的方法,深度模态混合网络(DMBN),通过混合多模态信号与随机加权机制,从机器人的多模态经验创建一个共同的潜在空间。我们首次证明,当深度学习与一种新的模态混合方案相结合时,可以促进动作识别并产生结构来维持解剖和基于效果的模仿能力。我们提出的系统,可以在任何时间步的任何期望的感觉/运动值的条件,并可以生成一个完整的多模态轨迹与期望的条件一致,并行避免预测误差的积累。我们进一步表明,给定来自不同视角的所需图像,即通过观察放置在桌子不同侧的其他机器人生成的图像,我们的系统可以生成对应于解剖学或基于效果的模仿行为的图像和关节角度序列。总的来说,建议的DMBN架构不仅作为一个计算模型,用于维持镜像神经元的能力,但也代表作为一个强大的机器学习架构的高维多模态时态数据与强大的检索能力与部分信息在一个或多个模态。

英文摘要:

Learning to interact with the environment not only empowers the agent with manipulation capability but also generates information to facilitate building of action understanding and imitation capabilities. This seems to be a strategy adopted by biological systems, in particular primates, as evidenced by the existence of mirror neurons that seem to be involved in multi-modal action understanding. How to benefit from the interaction experience of the robots to enable understanding actions and goals of other agents is still a challenging question. In this study, we propose a novel method, deep modality blending networks (DMBN), that creates a common latent space from multi-modal experience of a robot by blending multi-modal signals with a stochastic weighting mechanism. We show for the first time that deep learning, when combined with a novel modality blending scheme, can facilitate action recognition and produce structures to sustain anatomical and effect-based imitation capabilities. Our proposed system, can be conditioned on any desired sensory/motor value at any time-step, and can generate a complete multi-modal trajectory consistent with the desired conditioning in parallel avoiding accumulation of prediction errors. We further showed that given desired images from different perspectives, i.e. images generated by the observation of other robots placed on different sides of the table, our system could generate image and joint angle sequences that correspond to either anatomical or effect based imitation behavior. Overall, the proposed DMBN architecture not only serves as a computational model for sustaining mirror neuron-like capabilities, but also stands as a powerful machine learning architecture for high-dimensional multi-modal temporal data with robust retrieval capabilities operating with partial information in one or multiple modalities.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑