Similar to humans perceiving visual scenes as objects, Object-Centric Learning (OCL) can abstract dense images or videos into sparse object-level features. Transformer-based OCL handles complex textures well due to the decoding guidance of discrete representation, obtained by discretizing noisy features in image or video feature maps using template features from a codebook. However, treating features as minimal units overlooks their composing attributes, thus impeding model generalization; indexing features with natural numbers loses attribute-level commonalities and characteristics, thus diminishing heuristics for model convergence. We propose \textit{Grouped Discrete Representation} (GDR) to address these issues by grouping features into attributes and indexing them with tuple numbers. In extensive experiments across different query initializations, dataset modalities, and model architectures, GDR consistently improves convergence and generalizability. Visualizations show that our method effectively captures attribute-level information in features. The source code will be available upon acceptance.
Grouped Discrete Representation for Object-Centric Learning
Department of Electrical Engineering and Automation, Aalto University, Finland(艾尔沃斯大学电气工程与自动化系);Department of Computer Science, Aalto University, Finland(艾尔沃斯大学计算机科学系);Center for Machine Vision and Signal Analysis, University of Oulu, Finland(奥卢大学机器视觉与信号分析中心)
CLIP-Guided Adaptable Self-Supervised Learning for Human-Centric Visual Tasks
Key Laboratory of Intelligent Information Processing, Institute of Computing Technology (ICT), Chinese Academy of Sciences (CAS), and University of Chinese Academy of Sciences(智能信息处理重点实验室,计算技术研究所(ICT),中国科学院(CAS)及中国科学院大学)
Top-Down Guidance for Learning Object-Centric Representations
MAIS, Institute of Automation, Chinese Academy of Sciences(自动化研究所);School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学人工智能学院);CAIR, HKSIS, Chinese Academy of Sciences(中国科学院计算机研究所);School of Computer Science and Engineering, the Faculty of Innovation Engineering, M.U.S.T(澳门科技大学计算机科学与工程学院)
Prompt-Driven Dynamic Object-Centric Learning for Single Domain Generalization
College of Intelligence and Computing, Tianjin University(天津大学智能与计算学部);School of Electronic Engineering, Xidian University(西安电子科技大学电子工程学院);Peng Cheng Laboratory(鹏城实验室)
Vehicle-centric Perception via Multimodal Structured Pre-training
Information Materials and Intelligent Sensing Laboratory of Anhui Province(安徽省信息材料与智能感知实验室);Anhui Provincial Key Laboratory of Multimodal Cognitive Computation(安徽省多模态认知计算重点实验室);the School of Artificial Intelligence, Anhui University(安徽大学人工智能学院);School of Computer Science and Technology, Anhui University(安徽大学计算机科学与技术学院);Institute of Artificial Intelligence, Hefei Comprehensive National Science Center(合肥综合性国家科学中心人工智能研究院)
HAP: Structure-Aware Masked Image Modeling for Human-Centric Perception
Zhejiang University(浙江大学);Baidu VIS(百度视觉技术部);University of Science and Technology Beijing(北京科技大学);South China University of Technology(华南理工大学);Shanghai Jiao Tong University(上海交通大学);Jilin University(吉林大学)
Hallo-Live: Real-Time Streaming Joint Audio-Video Avatar Generation with Asynchronous Dual-Stream and Human-Centric Preference Distillation
Shanghai Innovation Institute(上海创新研究院);Fudan University(复旦大学);University of Science and Technology of China(中国科学技术大学);Nanjing University(南京大学);Baidu(百度)
Mila - Quebec AI Institute(米拉-魁北克人工智能研究所);Université de Montréal(蒙特利尔大学);University of Amsterdam(阿姆斯特丹大学);University of Tübingen(蒂宾根大学);Archimedes/Athena RC(阿基米德/雅典娜研究中心)