Unsupervised object-centric learning (OCL) decomposes visual scenes into distinct entities. Slot attention is a popular approach that represents individual objects as latent vectors, called slots. Current methods obtain these slot representations solely from the last layer of a pre-trained vision transformer (ViT), ignoring valuable, semantically rich information encoded across the other layers. To better utilize this latent semantic information, we introduce MUFASA, a lightweight plug-and-play framework for slot-attention-based approaches to unsupervised object segmentation. Our model computes slot attention across multiple feature layers of the ViT encoder, fully leveraging their semantic richness. We propose a fusion strategy to aggregate slots obtained on multiple layers into a unified object-centric representation. Integrating MUFASA into existing OCL methods improves their segmentation results across multiple datasets, setting a new state of the art while simultaneously improving training convergence with only minor inference overhead.
Attention Normalization Impacts Cardinality Generalization in Slot Attention
Max Planck Institute for Intelligent Systems（马克斯·普朗克智能系统研究所）；University of Augsburg（奥格斯堡大学）；Robert Bosch GmbH - Corporate Research（罗伯特·博世有限公司企业研究部）；Bosch Center for Artificial Intelligence (BCAI)（博世人工智能中心）
Slot Attention with Re-Initialization and Self-Distillation
Department of Electrical Engineering and Automation（电气工程与自动化系）；Aalto University（阿尔托大学）；Department of Computer Science（计算机科学系）；Center for Machine Vision and Signal Analysis（机器视觉与信号分析中心）；University of Oulu（奥卢大学）
本文提出 DIAS 方法，通过槽重初始化和自蒸馏减少冗余槽，提升对象为中心学习的对象发现与识别性能。
Slot Attention-based Feature Filtering for Few-Shot Learning
AIBA, Departament de Matemàtiques & Informàtica Universitat de Barcelona（AIBA，数学与信息学院巴塞罗那大学）；Departamento de Ingeniería de Sistemas y Computación Universidad Católica del Norte（系统与计算系卡罗林大学）；Institute of Neuroscience, Universitat de Barcelona（神经科学研究所巴塞罗那大学）
Explainable artificial intelligence has been gaining attention in the past few years. However, most existing methods are based on gradients or intermediate features, which are not directly involved in the decision-making process of the classifier. In this paper, we propose a slot attention-based classifier called SCOUTER for transparent yet accurate classification. Two major differences from other attention-based methods include: (a) SCOUTER's explanation is involved in the final confidence for each category, offering more intuitive interpretation, and (b) all the categories have their corresp…
Academy for Engineering and Technology, Fudan University（复旦大学工程与应用技术研究院）；Fudan Zhangjiang Institute（复旦张江研究院）；Sensetime Research（商汤科技研究院）；School of Computer Science, Fudan University（复旦大学计算机科学技术学院）；The Hong Kong Polytechnic University（香港理工大学）
It has been found that temporal action proposal generation, which aims to discover the temporal action instances within the range of the start and end frames in the untrimmed videos, can largely benefit from proper temporal and semantic context exploitation. The latest efforts were dedicated to considering the temporal context and similarity-based semantic contexts through self-attention modules. However, they still suffer from cluttered background information and limited contextual feature learning. In this paper, we propose a novel Pyramid Region-based Slot Attention (PRSlot) module to addre…
Predicting Video Slot Attention Queries from Random Slot-Feature Pairs
Department of Electrical Engineering and Automation, Aalto University（艾罗大学电气工程与自动化系）；Gaoling School of Artificial Intelligence, Renmin University of China（中国人民大学研究生院人工智能学院）；Department of Computer Science, Aalto University（艾罗大学计算机科学系）；Center for Machine Vision and Signal Analysis, University of Oulu（奥卢大学机器视觉与信号分析中心）
Vision-and-language navigation (VLN), a frontier study aiming to pave the way for general-purpose robots, has been a hot topic in the computer vision and natural language processing community. The VLN task requires an agent to navigate to a goal location following natural language instructions in unfamiliar environments. Recently, transformer-based models have gained significant improvements on the VLN task. Since the attention mechanism in the transformer architecture can better integrate inter- and intra-modal information of vision and language. However, there exist two problems in current t…
Unlocking Slot Attention by Changing Optimal Transport Costs
Samsung - SAIT AI Lab, Montreal（三星-三星先进技术研究院蒙特利尔人工智能实验室）；University of Amsterdam（阿姆斯特丹大学）；Mila, Université de Montreal（蒙特利尔大学米拉人工智能研究所）；TNO（荷兰应用科学研究组织）
Smoothing Slot Attention Iterations and Recurrences
Department of Electrical Engineering and Automation, Aalto University, Espoo, Finland（电气工程与自动化系，阿alto大学，埃斯波，芬兰）；Department of Computer Science, Aalto University, Espoo, Finland（计算机科学系，阿alto大学，埃斯波，芬兰）；Center for Machine Vision and Signal Analysis, University of Oulu, Oulu, Finland（机器视觉与信号分析中心，奥卢大学，奥卢，芬兰）
TSA: Temporal Slot Activation for Persistent Object-Centric Video Representation
University of Arkansas, USA（阿肯色大学）；Max Planck Research School for Intelligent Systems（马克斯·普朗克智能系统研究所）；Google Research, Google（谷歌研究院）；University of Liverpool, UK（利物浦大学）；Adobe Research（Adobe研究院）
ETH Zurich（苏黎世联邦理工学院）；Max-Planck Institute for Intelligent Systems（马克斯·普朗克智能系统研究所）；Google Research（谷歌研究院）
Learning object-centric representations of complex scenes is a promising step towards enabling efficient abstract reasoning from low-level perceptual features. Yet, most deep learning approaches learn distributed representations that do not capture the compositional properties of natural scenes. In this paper, we present the Slot Attention module, an architectural component that interfaces with perceptual representations such as the output of a convolutional neural network and produces a set of task-dependent abstract representations which we call slots. These slots are exchangeable and can bi…
FORLA: Federated Object-centric Representation Learning with Slot Attention
PCASO Laboratory, Dept. of Surgery, University of Pennsylvania（宾夕法尼亚大学外科部PCASO实验室）；Dept. of Computer and Information Science, University of Pennsylvania（宾夕法尼亚大学计算机与信息科学系）
Explainable Image Recognition via Enhanced Slot-attention Based Classifier
Institute of Datability Science (IDS), Osaka University（大阪大学数据科学研究所）；Premium Research Institute for Human Metaverse Medicine (WPI-PRIMe), Osaka University（大阪大学人类元宇宙医学高级研究院）
A Hyperbolic Perspective on Hierarchical Structure in Object-Centric Scene Representations
Aalborg University & Pioneer Centre for AI, Denmark（奥尔堡大学 & 人工智能先锋中心，丹麦）；Universitat de Barcelona & Computer Vision Center, Spain（巴塞罗那大学 & 计算机视觉中心，西班牙）；Milestone Systems, Denmark（Milestone Systems，丹麦）；University of Guelph & Vector Institute, Canada（圭尔夫大学 & 向量研究所，加拿大）
Future Slot Prediction for Unsupervised Object Discovery in Surgical Video
PCASO Laboratory, Department of Surgery, University of Pennsylvania（匹兹堡大学外科实验室、外科系、匹兹堡大学）；Department of Computer and Information Science, University of Pennsylvania（计算机与信息科学系、匹兹堡大学）
The ability to decompose complex natural scenes into meaningful object-centric abstractions lies at the core of human perception and reasoning. In the recent culmination of unsupervised object-centric learning, the Slot-Attention module has played an important role with its simple yet effective design and fostered many powerful variants. These methods, however, have been exceedingly difficult to train without supervision and are ambiguous in the notion of object, especially for complex natural scenes. In this paper, we propose to address these issues by investigating the potential of learnable…
