使用原力卢克学习通过模拟效果预测物理力
Use the Force, Luke! Learning to Predict Physical Forces by Simulating Effects
摘要:
当我们人类观看人与物体交互的视频时,我们不仅可以推断出发生了什么,甚至可以提取可操作的信息并模仿这些交互。另一方面,当前的识别或几何方法缺乏动作表示的物理性。在本文中,我们采取了一个更物理的理解行动。我们解决的问题,推断接触点和物理力的视频人类与物体的互动。解决这个问题的主要挑战之一是获得部队的地面实况标签。我们通过使用物理模拟器进行监督来回避这个问题。具体来说,我们使用模拟器来预测效果,并强制执行估计的力必须导致与视频中描述的相同的效果。我们的定量和定性结果表明:(a)我们可以从视频中预测有意义的力,其效果导致对观察到的运动的准确模仿;(b)通过联合优化接触点和力预测,与独立训练相比,我们可以提高这两项任务的性能;(c)我们可以从这个模型中学习一种表示,该模型使用Few Shot示例推广到新对象。
英文摘要:
When we humans look at a video of human-object interaction, we can not only infer what is happening but we can even extract actionable information and imitate those interactions. On the other hand, current recognition or geometric approaches lack the physicality of action representation. In this paper, we take a step towards a more physical understanding of actions. We address the problem of inferring contact points and the physical forces from videos of humans interacting with objects. One of the main challenges in tackling this problem is obtaining ground-truth labels for forces. We sidestep this problem by instead using a physics simulator for supervision. Specifically, we use a simulator to predict effects and enforce that estimated forces must lead to the same effect as depicted in the video. Our quantitative and qualitative results show that (a) we can predict meaningful forces from videos whose effects lead to accurate imitation of the motions observed, (b) by jointly optimizing for contact point and force prediction, we can improve the performance on both tasks in comparison to independent training, and (c) we can learn a representation from this model that generalizes to novel objects using few shot examples.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。