学习语音控制机器人的视觉-音频表示
Learning Visual-Audio Representations for Voice-Controlled Robots
摘要:
基于表示学习的最新进展,我们提出了一种新的面向任务的语音控制机器人与原始传感器输入的管道。以前的方法依赖于大量的标签和特定于任务的奖励函数。这种方法不仅在部署后很难改进,而且在机器人平台和任务之间的推广也有限。为了解决这些问题,我们的管道首先学习了一种将图像和声音命令相关联的视觉-音频表示(VAR)。然后,机器人通过使用VAR生成的奖励的强化学习来学习完成声音命令。我们用各种声音类型、机器人和任务来展示我们的方法。我们表明,我们的方法优于以前的工作少得多的标签。我们在模拟和真实世界的实验中表明,系统可以在以前看不见的情况下自我改进,给出合理数量的新标记数据。
英文摘要:
Based on the recent advancements in representation learning, we propose a novel pipeline for task-oriented voice-controlled robots with raw sensor inputs. Previous methods rely on a large number of labels and task-specific reward functions. Not only can such an approach hardly be improved after the deployment, but also has limited generalization across robotic platforms and tasks. To address these problems, our pipeline first learns a visual-audio representation (VAR) that associates images and sound commands. Then the robot learns to fulfill the sound command via reinforcement learning using the reward generated by the VAR. We demonstrate our approach with various sound types, robots, and tasks. We show that our method outperforms previous work with much fewer labels. We show in both the simulated and real-world experiments that the system can self-improve in previously unseen scenarios given a reasonable number of newly labeled data.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。