arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2009.08292cs.CVcs.LGcs.RO
收藏

利用微分物理学从视频中学习识别物理参数

Learning to Identify Physical Parameters from Video Using Differentiable Physics

University of Siegen(锡根大学);Max Planck Institute for Intelligent Systems(马克斯·普朗克智能系统研究所)

Rama Krishna Kandukuri, Jan Achterhold, Michael Möller, Jörg Stückler

摘要:

视频表示学习由于其在活动和场景预测或基于视觉的规划和控制中的应用,最近在计算机视觉中引起了关注。视频预测模型通常学习从输入帧编码并解码回图像的视频的潜在表示。即使以动作为条件,纯粹基于深度学习的架构通常也缺乏物理上可解释的潜在空间。在这项研究中,我们使用一个可微的物理引擎内的动作条件视频表示网络学习物理潜在的表示。我们提出了监督和自监督学习方法来训练我们的网络和识别物理特性。后者使用空间Transformers将物理状态解码回图像。在我们的实验中的模拟场景包括推动,滑动和碰撞的物体,我们还分析了物理属性的可观测性。在实验中,我们证明了我们的网络可以学习编码图像,并识别模拟场景中视频和动作序列的质量和摩擦等物理属性。我们评估我们的监督和自我监督的方法的准确性,并将其与直接从状态轨迹学习的系统辨识基线进行比较。我们还展示了我们的方法从输入图像和动作预测未来视频帧的能力。

英文摘要:

Video representation learning has recently attracted attention in computer vision due to its applications for activity and scene forecasting or vision-based planning and control. Video prediction models often learn a latent representation of video which is encoded from input frames and decoded back into images. Even when conditioned on actions, purely deep learning based architectures typically lack a physically interpretable latent space. In this study, we use a differentiable physics engine within an action-conditional video representation network to learn a physical latent representation. We propose supervised and self-supervised learning methods to train our network and identify physical properties. The latter uses spatial transformers to decode physical states back into images. The simulation scenarios in our experiments comprise pushing, sliding and colliding objects, for which we also analyze the observability of the physical properties. In experiments we demonstrate that our network can learn to encode images and identify physical properties like mass and friction from videos and action sequences in the simulated scenarios. We evaluate the accuracy of our supervised and self-supervised methods and compare it with a system identification baseline which directly learns from state trajectories. We also demonstrate the ability of our method to predict future video frames from input images and actions.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑