Object-centric learning aims to decompose an input image into a set of meaningful object files (slots). These latent object representations enable a variety of downstream tasks. Yet, object-centric learning struggles on real-world datasets, which contain multiple objects of complex textures and shapes in natural everyday scenes. To address this, we introduce Guided Latent Slot Diffusion (GLASS), a novel slot-attention model that learns in the space of generated images and uses semantic and instance guidance modules to learn better slot embeddings for various downstream tasks. Our experiments show that GLASS surpasses state-of-the-art slot-attention methods by a wide margin on tasks such as (zero-shot) object discovery and conditional image generation for real-world scenes. Moreover, GLASS enables the first application of slot attention to the compositional generation of complex, realistic scenes.
Although DALL-E has shown an impressive ability of composition-based systematic generalization in image generation, it requires the dataset of text-image pairs and the compositionality is provided by the text. In contrast, object-centric representation models like the Slot Attention model learn composable representations without the text prompt. However, unlike DALL-E its ability to systematically generalize for zero-shot generation is significantly limited. In this paper, we propose a simple but novel slot-based autoencoding architecture, called SLATE, for combining the best of both worlds: l…
ETH Zurich（苏黎世联邦理工学院）；Max-Planck Institute for Intelligent Systems（马克斯·普朗克智能系统研究所）；Google Research（谷歌研究院）
Learning object-centric representations of complex scenes is a promising step towards enabling efficient abstract reasoning from low-level perceptual features. Yet, most deep learning approaches learn distributed representations that do not capture the compositional properties of natural scenes. In this paper, we present the Slot Attention module, an architectural component that interfaces with perceptual representations such as the output of a convolutional neural network and produces a set of task-dependent abstract representations which we call slots. These slots are exchangeable and can bi…
FORLA: Federated Object-centric Representation Learning with Slot Attention
PCASO Laboratory, Dept. of Surgery, University of Pennsylvania（宾夕法尼亚大学外科部PCASO实验室）；Dept. of Computer and Information Science, University of Pennsylvania（宾夕法尼亚大学计算机与信息科学系）
University of Amsterdam（阿姆斯特丹大学）；Google Research（谷歌研究院）
Contrastive, self-supervised learning of object representations recently emerged as an attractive alternative to reconstruction-based training. Prior approaches focus on contrasting individual object representations (slots) against one another. However, a fundamental problem with this approach is that the overall contrastive loss is the same for (i) representing a different object in each slot, as it is for (ii) (re-)representing the same object in all slots. Thus, this objective does not inherently push towards the emergence of object-centric representations in the slots. We address this prob…
Samsung Research China - Beijing (SRC-B)（三星中国研究院（北京））；Samsung Advanced Institute of Technology (SAIT)（三星先进技术研究院）；University of Oxford（牛津大学）；The University of Hong Kong（香港大学）
Video Panoptic Segmentation (VPS) aims at assigning a class label to each pixel, uniquely segmenting and identifying all object instances consistently across all frames. Classic solutions usually decompose the VPS task into several sub-tasks and utilize multiple surrogates (e.g. boxes and masks, centres and offsets) to represent objects. However, this divide-and-conquer strategy requires complex post-processing in both spatial and temporal domains and is vulnerable to failures from surrogate tasks. In this paper, inspired by object-centric learning which learns compact and robust object repres…
University of Toronto（多伦多大学）；Vector Institute（向量研究所）；Samsung AI Centre Toronto（三星多伦多人工智能中心）；Google Research（谷歌研究院）
Understanding dynamics from visual observations is a challenging problem that requires disentangling individual objects from the scene and learning their interactions. While recent object-centric models can successfully decompose a scene into objects, modeling their dynamics effectively still remains a challenge. We address this problem by introducing SlotFormer -- a Transformer-based autoregressive model operating on learned object-centric representations. Given a video clip, our approach reasons over object features to model spatio-temporal relationships and predicts accurate future object s…
Object-Centric Temporal Consistency via Conditional Autoregressive Inductive Biases
Delft University of Technology（代尔夫特理工大学）；The University of Tokyo（东京大学）；Mila, University of Montreal（蒙特利尔大学米拉研究所）；Google DeepMind（谷歌DeepMind）；Agency for Science, Technology and Research（新加坡科技研究局）；Nanyang Technological University（南洋理工大学）
Unlocking Slot Attention by Changing Optimal Transport Costs
Samsung - SAIT AI Lab, Montreal（三星-三星先进技术研究院蒙特利尔人工智能实验室）；University of Amsterdam（阿姆斯特丹大学）；Mila, Université de Montreal（蒙特利尔大学米拉人工智能研究所）；TNO（荷兰应用科学研究组织）
The ability to decompose complex natural scenes into meaningful object-centric abstractions lies at the core of human perception and reasoning. In the recent culmination of unsupervised object-centric learning, the Slot-Attention module has played an important role with its simple yet effective design and fostered many powerful variants. These methods, however, have been exceedingly difficult to train without supervision and are ambiguous in the notion of object, especially for complex natural scenes. In this paper, we propose to address these issues by investigating the potential of learnable…
Mila - Quebec AI Institute（米拉-魁北克人工智能研究所）；Université de Montréal（蒙特利尔大学）；University of Amsterdam（阿姆斯特丹大学）；University of Tübingen（蒂宾根大学）；Archimedes/Athena RC（阿基米德/雅典娜研究中心）
TSA: Temporal Slot Activation for Persistent Object-Centric Video Representation
University of Arkansas, USA（阿肯色大学）；Max Planck Research School for Intelligent Systems（马克斯·普朗克智能系统研究所）；Google Research, Google（谷歌研究院）；University of Liverpool, UK（利物浦大学）；Adobe Research（Adobe研究院）
A Hyperbolic Perspective on Hierarchical Structure in Object-Centric Scene Representations
Aalborg University & Pioneer Centre for AI, Denmark（奥尔堡大学 & 人工智能先锋中心，丹麦）；Universitat de Barcelona & Computer Vision Center, Spain（巴塞罗那大学 & 计算机视觉中心，西班牙）；Milestone Systems, Denmark（Milestone Systems，丹麦）；University of Guelph & Vector Institute, Canada（圭尔夫大学 & 向量研究所，加拿大）
University of Hong Kong（香港大学）；MEGVII Technology（旷视科技）；University of Edinburgh（爱丁堡大学）；LunarAI
In this paper, we tackle the problem of learning visual representations from unlabeled scene-centric data. Existing works have demonstrated the potential of utilizing the underlying complex structure within scene-centric data; still, they commonly rely on hand-crafted objectness priors or specialized pretext tasks to build a learning framework, which may harm generalizability. Instead, we propose contrastive learning from data-driven semantic slots, namely SlotCon, for joint semantic grouping and representation learning. The semantic grouping is performed by assigning pixels to a set of learna…
