体验嵌入式视觉前景
Experience-Embedded Visual Foresight
摘要:
视觉预见为智能体提供了一个未来的窗口,它可以用来在事件发生之前预测事件并计划战略行为。虽然在受限环境下的视频预测方面取得了令人印象深刻的结果,但这些模型在面对不熟悉的现实世界对象时无法推广。在本文中,我们通过快速自适应来解决泛化问题,其中我们训练预测模型以快速适应观察到的新对象的视觉动态。我们的方法,经验嵌入式视觉模糊(EVF),共同学习一个快速适应模块,它将观察到的新对象的轨迹编码到一个向量嵌入,和一个视觉预测模型,它的条件下,这种嵌入生成物理上合理的预测。为了进行评估,我们将我们的方法与视频预测的基线进行比较,并将其在两个真实控制任务上的实用性进行基准测试。我们表明,我们的方法能够快速适应新的视觉动态,并在操纵新对象时实现比基线更低的误差。
英文摘要:
Visual foresight gives an agent a window into the future, which it can use to anticipate events before they happen and plan strategic behavior. Although impressive results have been achieved on video prediction in constrained settings, these models fail to generalize when confronted with unfamiliar real-world objects. In this paper, we tackle the generalization problem via fast adaptation, where we train a prediction model to quickly adapt to the observed visual dynamics of a novel object. Our method, Experience-embedded Visual Foresight (EVF), jointly learns a fast adaptation module, which encodes observed trajectories of the new object into a vector embedding, and a visual prediction model, which conditions on this embedding to generate physically plausible predictions. For evaluation, we compare our method against baselines on video prediction and benchmark its utility on two real-world control tasks. We show that our method is able to quickly adapt to new visual dynamics and achieves lower error than the baselines when manipulating novel objects.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。