Object-centric learning (OCL) aspires general and compositional understanding of scenes by representing a scene as a collection of object-centric representations. OCL has also been extended to multi-view image and video datasets to apply various data-driven inductive biases by utilizing geometric or temporal information in the multi-image data. Single-view images carry less information about how to disentangle a given scene than videos or multi-view images do. Hence, owing to the difficulty of applying inductive biases, OCL for single-view images remains challenging, resulting in inconsistent learning of object-centric representation. To this end, we introduce a novel OCL framework for single-view images, SLot Attention via SHepherding (SLASH), which consists of two simple-yet-effective modules on top of Slot Attention. The new modules, Attention Refining Kernel (ARK) and Intermediate Point Predictor and Encoder (IPPE), respectively, prevent slots from being distracted by the background noise and indicate locations for slots to focus on to facilitate learning of object-centric representation. We also propose a weak semi-supervision approach for OCL, whilst our proposed framework can be used without any assistant annotation during the inference. Experiments show that our proposed method enables consistent learning of object-centric representation and achieves strong performance across four datasets. Code is available at \url{https://github.com/object-understanding/SLASH}.
TSA: Temporal Slot Activation for Persistent Object-Centric Video Representation
University of Arkansas, USA（阿肯色大学）；Max Planck Research School for Intelligent Systems（马克斯·普朗克智能系统研究所）；Google Research, Google（谷歌研究院）；University of Liverpool, UK（利物浦大学）；Adobe Research（Adobe研究院）
A Hyperbolic Perspective on Hierarchical Structure in Object-Centric Scene Representations
Aalborg University & Pioneer Centre for AI, Denmark（奥尔堡大学 & 人工智能先锋中心，丹麦）；Universitat de Barcelona & Computer Vision Center, Spain（巴塞罗那大学 & 计算机视觉中心，西班牙）；Milestone Systems, Denmark（Milestone Systems，丹麦）；University of Guelph & Vector Institute, Canada（圭尔夫大学 & 向量研究所，加拿大）
Future Slot Prediction for Unsupervised Object Discovery in Surgical Video
PCASO Laboratory, Department of Surgery, University of Pennsylvania（匹兹堡大学外科实验室、外科系、匹兹堡大学）；Department of Computer and Information Science, University of Pennsylvania（计算机与信息科学系、匹兹堡大学）
ETH Zurich（苏黎世联邦理工学院）；Max-Planck Institute for Intelligent Systems（马克斯·普朗克智能系统研究所）；Google Research（谷歌研究院）
Learning object-centric representations of complex scenes is a promising step towards enabling efficient abstract reasoning from low-level perceptual features. Yet, most deep learning approaches learn distributed representations that do not capture the compositional properties of natural scenes. In this paper, we present the Slot Attention module, an architectural component that interfaces with perceptual representations such as the output of a convolutional neural network and produces a set of task-dependent abstract representations which we call slots. These slots are exchangeable and can bi…
Internalizing Temporal Consistency in Video Object-Centric Learning without Explicit Regularization
Department of Electrical Engineering and Automation, Aalto University, Finland（艾尔沃斯大学电气工程与自动化系）；Department of Computer Science, Aalto University, Finland（艾尔沃斯大学计算机科学系）；Center for Machine Vision and Signal Analysis, University of Oulu, Finland（奥卢大学机器视觉与信号分析中心）
Cycle Consistency in Video Object-Centric Learning
Department of Electrical Engineering and Automation, Aalto University, Espoo, Finland（艾洛大学电气工程与自动化系，芬兰 Espoo）；School of Artificial Intelligence and Automation, Huazhong University of Science and Technology, Wuhan, China（华中科技大学人工智能与自动化学院，中国 Wuhan）；Department of Computer Science, Aalto University, Espoo, Finland（艾洛大学计算机科学系，芬兰 Espoo）；Center for Machine Vision and Signal Analysis, University of Oulu, Oulu, Finland（奥卢大学机器视觉与信号分析中心，芬兰 Oulu）
Attention Normalization Impacts Cardinality Generalization in Slot Attention
Max Planck Institute for Intelligent Systems（马克斯·普朗克智能系统研究所）；University of Augsburg（奥格斯堡大学）；Robert Bosch GmbH - Corporate Research（罗伯特·博世有限公司企业研究部）；Bosch Center for Artificial Intelligence (BCAI)（博世人工智能中心）
University of Toronto（多伦多大学）；Vector Institute（向量研究所）；Samsung AI Centre Toronto（三星多伦多人工智能中心）；Google Research（谷歌研究院）
Understanding dynamics from visual observations is a challenging problem that requires disentangling individual objects from the scene and learning their interactions. While recent object-centric models can successfully decompose a scene into objects, modeling their dynamics effectively still remains a challenge. We address this problem by introducing SlotFormer -- a Transformer-based autoregressive model operating on learned object-centric representations. Given a video clip, our approach reasons over object features to model spatio-temporal relationships and predicts accurate future object s…
FORLA: Federated Object-centric Representation Learning with Slot Attention
PCASO Laboratory, Dept. of Surgery, University of Pennsylvania（宾夕法尼亚大学外科部PCASO实验室）；Dept. of Computer and Information Science, University of Pennsylvania（宾夕法尼亚大学计算机与信息科学系）
University of Amsterdam（阿姆斯特丹大学）；Google Research（谷歌研究院）
Contrastive, self-supervised learning of object representations recently emerged as an attractive alternative to reconstruction-based training. Prior approaches focus on contrasting individual object representations (slots) against one another. However, a fundamental problem with this approach is that the overall contrastive loss is the same for (i) representing a different object in each slot, as it is for (ii) (re-)representing the same object in all slots. Thus, this objective does not inherently push towards the emergence of object-centric representations in the slots. We address this prob…
SlotVLA: Towards Modeling of Object-Relation Representations in Robotic Manipulation
University of Arkansas（亚拉巴马大学）；FPT Software AI Center（FPT软件人工智能中心）；University of Stuttgart（斯图加特大学）；Aalborg University（奥尔堡大学）；Carnegie Mellon University（卡内基梅隆大学）；University of Liverpool（利物浦大学）；German Research Center for Artificial Intelligence（德国人工智能研究中心）；Max Planck Research School for Intelligent Systems（马克斯·普朗克智能系统研究学校）
Samsung Research China - Beijing (SRC-B)（三星中国研究院（北京））；Samsung Advanced Institute of Technology (SAIT)（三星先进技术研究院）；University of Oxford（牛津大学）；The University of Hong Kong（香港大学）
Video Panoptic Segmentation (VPS) aims at assigning a class label to each pixel, uniquely segmenting and identifying all object instances consistently across all frames. Classic solutions usually decompose the VPS task into several sub-tasks and utilize multiple surrogates (e.g. boxes and masks, centres and offsets) to represent objects. However, this divide-and-conquer strategy requires complex post-processing in both spatial and temporal domains and is vulnerable to failures from surrogate tasks. In this paper, inspired by object-centric learning which learns compact and robust object repres…
Slot-BERT: Self-supervised Object Discovery in Surgical Video
Outcomes Laboratory, Department of Surgery, University of Pennsylvania, Philadelphia, PA, USA.；Department of Computer；Information Science, University of Pennsylvania, Philadelphia, PA, USA.；Division of Thoracic Surgery, Toronto General Hospital, University Health Network, Toronto, Ontario, Canada.；Surgical Artificial Intelligence Research Academy, University Health Network, Toronto, ON, Canada.
