arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2101.02692cs.CVcs.RO
收藏

Where2Act:从像素到三维铰接物体的动作

Where2Act: From Pixels to Actions for Articulated 3D Objects

Stanford University(斯坦福大学);Facebook AI Research(Facebook人工智能研究)

Kaichun Mo, Leonidas Guibas, Mustafa Mukadam, Abhinav Gupta, Shubham Tulsiani

摘要:

视觉感知的基本目标之一是允许智能体与其环境进行有意义的交互。在本文中,我们朝着这个长期目标迈出了一步-我们提取了与基本动作相关的高度本地化的可操作信息,例如推或拉具有可移动部件的铰接对象。例如,给定一个抽屉,我们的网络预测在把手上施加拉力会打开抽屉。我们提出,讨论和评估新的网络架构,给定图像和深度数据,预测每个像素可能的动作集,以及可能在力作用下移动的关节部位的区域。我们提出了一个学习互动框架与在线数据采样策略,使我们能够训练网络模拟(SAPIEN)和跨类别的推广。查看网站以获取代码和数据发布:https://cs.stanford.edu/~kaichun/where2act/

英文摘要:

One of the fundamental goals of visual perception is to allow agents to meaningfully interact with their environment. In this paper, we take a step towards that long-term goal -- we extract highly localized actionable information related to elementary actions such as pushing or pulling for articulated objects with movable parts. For example, given a drawer, our network predicts that applying a pulling force on the handle opens the drawer. We propose, discuss, and evaluate novel network architectures that given image and depth data, predict the set of actions possible at each pixel, and the regions over articulated parts that are likely to move under the force. We propose a learning-from-interaction framework with an online data sampling strategy that allows us to train the network in simulation (SAPIEN) and generalizes across categories. Check the website for code and data release: https://cs.stanford.edu/~kaichun/where2act/

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑