后见之明：来自物理相互作用的无监督结构动力学模型
Hindsight for Foresight: Unsupervised Structured Dynamics Models from Physical Interaction
摘要：
智能体学习与世界交互的一个关键挑战是推理物体的物理属性，并预测它们在外力作用下的动态。为了通过交互将学习扩展到许多对象和场景，机器人应该能够在不需要人类监督的情况下从现实世界的经验中提高自己的性能。为此，我们提出了一种新的方法，直接从未标记的3D点云和图像建模的动态机器人的相互作用。与以前的方法不同，我们的方法不需要跟踪器或任何预先训练的感知网络提供的地面实况数据关联。为了从未标记的真实世界交互数据中学习，我们强制执行估计的3D云，动作和2D图像与观察到的一致性。我们的联合正向和反向网络学习将场景分割成显着的对象部分，并在应用动作的影响下预测它们的3D运动。此外，我们的对象为中心的模型输出动作条件的三维场景流，对象掩模和二维光流作为紧急属性。我们在模拟和现实世界的数据进行了广泛的评估，表明我们的配方导致有效的，可解释的模型，可用于可视化控制和规划。视频、代码和数据集可在http://hind4sight.cs.uni-freiburg.de上获得
英文摘要：
A key challenge for an agent learning to interact with the world is to reason about physical properties of objects and to foresee their dynamics under the effect of applied forces. In order to scale learning through interaction to many objects and scenes, robots should be able to improve their own performance from real-world experience without requiring human supervision. To this end, we propose a novel approach for modeling the dynamics of a robot's interactions directly from unlabeled 3D point clouds and images. Unlike previous approaches, our method does not require ground-truth data associations provided by a tracker or any pre-trained perception network. To learn from unlabeled real-world interaction data, we enforce consistency of estimated 3D clouds, actions and 2D images with observed ones. Our joint forward and inverse network learns to segment a scene into salient object parts and predicts their 3D motion under the effect of applied actions. Moreover, our object-centric model outputs action-conditioned 3D scene flow, object masks and 2D optical flow as emergent properties. Our extensive evaluation both in simulation and with real-world data demonstrates that our formulation leads to effective, interpretable models that can be used for visuomotor control and planning. Videos, code and dataset are available at http://hind4sight.cs.uni-freiburg.de
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
