狂欢观看：从情景喜剧中学习的可扩展性
Binge Watching: Scaling Affordance Learning from Sitcoms
摘要：
近年来，人们对感知和动作的联合建模重新产生了兴趣。这项调查的核心是建模示能的想法（示能是场景中交互的机会。换句话说，它表示对象可以用于哪些操作）。然而，当涉及到预测启示时，即使是最先进的方法仍然不使用任何ConvNets。为什么会这样？与语义或3D任务不同，仍然不存在任何大规模的可供性数据集。在本文中，我们解决了创建学习启示的最大数据集之一的挑战。我们使用七个情景喜剧提取一组不同的场景，以及演员如何与场景中的不同对象互动。我们的数据集包括超过10K个场景和28K种人类可以与这些10K图像交互的方式。我们还提出了一个两步的方法来预测启示在一个新的场景。在第一步中，给定场景中的位置，我们对30个姿势类别中的哪一个分类为可能的示能表示姿势。给定姿态类和场景，然后使用变分自编码器（VAE）来提取姿态的比例和变形。VAE允许我们在测试时对可能的姿势分布进行采样。最后，我们展示了大规模数据在学习可推广和强大的启示模型中的重要性。
英文摘要：
In recent years, there has been a renewed interest in jointly modeling perception and action. At the core of this investigation is the idea of modeling affordances(Affordances are opportunities of interaction in the scene. In other words, it represents what actions can the object be used for). However, when it comes to predicting affordances, even the state of the art approaches still do not use any ConvNets. Why is that? Unlike semantic or 3D tasks, there still does not exist any large-scale dataset for affordances. In this paper, we tackle the challenge of creating one of the biggest dataset for learning affordances. We use seven sitcoms to extract a diverse set of scenes and how actors interact with different objects in the scenes. Our dataset consists of more than 10K scenes and 28K ways humans can interact with these 10K images. We also propose a two-step approach to predict affordances in a new scene. In the first step, given a location in the scene we classify which of the 30 pose classes is the likely affordance pose. Given the pose class and the scene, we then use a Variational Autoencoder (VAE) to extract the scale and deformation of the pose. The VAE allows us to sample the distribution of possible poses at test time. Finally, we show the importance of large-scale data in learning a generalizable and robust model of affordances.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
