arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2007.04954cs.CVcs.GRcs.LGcs.RO
收藏

ThreeDWorld:交互式多模态物理仿真平台

ThreeDWorld: A Platform for Interactive Multi-Modal Physical Simulation

MIT-IBM Watson AI Lab(麻省理工学院-IBM沃森人工智能实验室);MIT(麻省理工学院);Stanford University(斯坦福大学);Harvard University(哈佛大学)
Chuang Gan, Jeremy Schwartz, Seth Alter, Damian Mrowca, Martin Schrimpf, James Traer, Julian De Freitas, Jonas Kubilius, Abhishek Bhandwaldar, Nick Haber, Megum… 展开作者

Chuang Gan, Jeremy Schwartz, Seth Alter, Damian Mrowca, Martin Schrimpf, James Traer, Julian De Freitas, Jonas Kubilius, Abhishek Bhandwaldar, Nick Haber, Megumi Sano, Kuno Kim, Elias Wang, Michael Lingelbach, Aidan Curtis, Kevin Feigelis, Daniel M. Bear, Dan Gutfreund, David Cox, Antonio Torralba, James J. DiCarlo, Joshua B. Tenenbaum, Josh H. McDermott, Daniel L. K. Yamins

摘要:

我们介绍了三个世界(TDW),交互式多模态物理仿真平台。TDW能够模拟高保真的感觉数据和丰富的3D环境中的移动代理和对象之间的物理交互。独特的属性包括:实时近照片逼真的图像渲染;对象和环境库及其定制例程;高效构建新环境类的生成程序;高保真音频渲染;各种材料类型的真实物理交互,包括布料,液体和可变形对象;体现人工智能智能体的可定制代理;以及支持人类与VR设备的交互。TDW的API使多个代理能够在模拟中进行交互,并返回一系列代表世界状态的传感器和物理数据。我们提出了在计算机视觉,机器学习和认知科学的新兴研究方向,包括多模态物理场景理解,物理动力学预测,多智能体交互,像孩子一样学习的模型,以及人类和神经网络的注意力研究的TDW启用的初步实验。

英文摘要:

We introduce ThreeDWorld (TDW), a platform for interactive multi-modal physical simulation. TDW enables simulation of high-fidelity sensory data and physical interactions between mobile agents and objects in rich 3D environments. Unique properties include: real-time near-photo-realistic image rendering; a library of objects and environments, and routines for their customization; generative procedures for efficiently building classes of new environments; high-fidelity audio rendering; realistic physical interactions for a variety of material types, including cloths, liquid, and deformable objects; customizable agents that embody AI agents; and support for human interactions with VR devices. TDW's API enables multiple agents to interact within a simulation and returns a range of sensor and physics data representing the state of the world. We present initial experiments enabled by TDW in emerging research directions in computer vision, machine learning, and cognitive science, including multi-modal physical scene understanding, physical dynamics predictions, multi-agent interactions, models that learn like a child, and attention studies in humans and neural networks.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑