arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2004.14638cs.ROcs.CV
收藏

走向场景描写

Towards Embodied Scene Description

Tsinghua University(清华大学)

Sinan Tan, Huaping Liu, Di Guo, Xinyu Zhang, Fuchun Sun

摘要:

具身性是所有智能主体(生物和机器人)的重要特征,而现有的场景描述任务主要集中在被动地分析图像,场景的语义理解与主体与环境的交互分离。在这项工作中,我们提出了嵌入式场景描述,它利用代理的体现能力,找到一个最佳的视角,在其环境中的场景描述任务。建立了一个以模仿学习和强化学习为范式的学习框架,用于指导智能体产生相应的感觉运动活动。该框架在AI2Thor数据集和真实世界机器人平台上进行了测试,证明了所开发方法的有效性和可扩展性。

英文摘要:

Embodiment is an important characteristic for all intelligent agents (creatures and robots), while existing scene description tasks mainly focus on analyzing images passively and the semantic understanding of the scenario is separated from the interaction between the agent and the environment. In this work, we propose the Embodied Scene Description, which exploits the embodiment ability of the agent to find an optimal viewpoint in its environment for scene description tasks. A learning framework with the paradigms of imitation learning and reinforcement learning is established to teach the intelligent agent to generate corresponding sensorimotor activities. The proposed framework is tested on both the AI2Thor dataset and a real world robotic platform demonstrating the effectiveness and extendability of the developed method.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑