Objects are usually associated with multiple attributes, and these attributes often exhibit high correlations. Modeling complex relationships between attributes poses a great challenge for multi-attribute learning. This paper proposes a simple yet generic framework named Label2Label to exploit the complex attribute correlations. Label2Label is the first attempt for multi-attribute prediction from the perspective of language modeling. Specifically, it treats each attribute label as a "word" describing the sample. As each sample is annotated with multiple attribute labels, these "words" will naturally form an unordered but meaningful "sentence", which depicts the semantic information of the corresponding sample. Inspired by the remarkable success of pre-training language models in NLP, Label2Label introduces an image-conditioned masked language model, which randomly masks some of the "word" tokens from the label "sentence" and aims to recover them based on the masked "sentence" and the context conveyed by image features. Our intuition is that the instance-wise attribute relations are well grasped if the neural net can infer the missing attributes based on the context and the remaining attribute hints. Label2Label is conceptually simple and empirically powerful. Without incorporating task-specific prior knowledge and highly specialized network designs, our approach achieves state-of-the-art results on three different multi-attribute learning tasks, compared to highly customized domain-specific methods. Code is available at https://github.com/Li-Wanhua/Label2Label.
Attribute-Centric Vision-Language Alignment for Fine-Grained Gait Understanding
School of Artificial Intelligence, Beijing Normal University(北京师范大学人工智能学院);The Hong Kong University of Science and Technology(香港科技大学);Zhejiang University(浙江大学);School of Artificial Intelligence, Shenzhen University(深圳大学人工智能学院);Southern University of Science and Technology(南方科技大学);Watrix Technology Limited Co. Ltd.(华瑞芯科技有限公司)
Vision-Language Attribute Disentanglement and Reinforcement for Lifelong Person Re-Identification
Wangxuan Institute of Computer Technology, Peking University, Beijing, China(北京大学王轩计算机技术研究院,北京,中国);University of Chinese Academy of Sciences, Beijing, China(中国科学院大学,北京,中国);School of Artificial Intelligence and Automation, Huazhong University of Science and Technology, Wuhan, China(华中科技大学人工智能与自动化学院,武汉,中国)
Southeast University(东南大学);Advanced Ocean Institute of Southeast University(东南大学先进海洋研究院);Northwestern Polytechnical University(西北工业大学);Binjiang Institute of Artificial Intelligence, ZJUT(浙江工业大学滨江人工智能研究院);Alibaba Group(阿里巴巴集团)
Attribute-Grounded Selective Reasoning for Artwork Emotion Understanding with Multimodal Large Language Models
Department of Computer Science and Technology, Jilin University(吉林大学计算机科学与技术学院);Department of Computer Science, National Taiwan University(国立台湾大学计算机科学系)
CASA: Class-Agnostic Shared Attributes in Vision-Language Models for Efficient Incremental Object Detection
School of Electronic and Computer Engineering, Peking University(北京大学电子与计算机工程学院);Peng Cheng Laboratory(鹏城实验室);School of Computer Science, Peking University(北京大学计算机学院)
Attribute-formed Class-specific Concept Space: Endowing Language Bottleneck Model with Better Interpretability and Scalability
University of Electronic Science and Technology of China(电子科技大学);Southwest Jiaotong University(西南交通大学);University of Minnesota(明尼苏达大学);Harvard University(哈佛大学);Nanyang Technological University(南洋理工大学)
Vision-to-Language Tasks Based on Attributes and Attention Mechanism
Northwestern Polytechnical University(西北工业大学);Xi’an Institute of Optics and Precision Mechanics, Chinese Academy of Sciences(中国科学院西安光学精密机械研究所);University of Chinese Academy of Sciences(中国科学院大学)
The Eye of Sherlock Holmes: Uncovering User Private Attribute Profiling via Vision-Language Model Agentic Framework
Nanyang Technological University(南洋理工大学);Beijing University of Technology(北京理工大学);Hengxin Tech(恒心科技);Alibaba Group(阿里巴巴集团);Squirrel Ai Learning(squirrel AI 学习)
VLM-PAR: A Vision Language Model for Pedestrian Attribute Recognition
Department of Innovation Engineering(创新工程系);University of Salento, Italy(意大利萨伦托大学);Institute of Applied Sciences and Intelligent Systems - CNR(应用科学与智能系统研究所 - CNR);University of the Basque Country UPV/EHU(巴斯克国家大学UPV/EHU);IKERBASQUE, Basque Foundation for Science(伊基塔斯克巴塞克基金会);Sorbonne University Abu Dhabi(索邦大学阿布扎比分校)
Pedestrian Attribute Recognition via CLIP based Prompt Vision-Language Fusion
School of Computer Science and Technology, Anhui University(安徽大学计算机科学与技术学院);Institute of Artificial Intelligence, Hefei Comprehensive National Science Center(合肥综合性国家科学中心人工智能研究院);Information Materials and Intelligent Sensing Laboratory of Anhui Province(安徽省信息材料与智能感知实验室);Anhui Provincial Key Laboratory of Multimodal Cognitive Computation(安徽省多模态认知计算重点实验室);School of Artificial Intelligence, Anhui University(安徽大学人工智能学院);Video Investigation Detachment of Hefei Public Security Bureau(合肥市公安局视频侦查支队)
Diagnosing Dense Same-Class Attribute Misbinding in Large Vision-Language Models
Qilu University of Technology (Shandong Academy of Sciences)(齐鲁工业大学(山东省科学院));China Telecom Digital Intelligence Technology Co., Ltd.(中国电信数字智能科技有限公司);Shenyang Aerospace University(沈阳航空航天大学);University of Nottingham Ningbo China(宁波诺丁汉大学)
Dual Attribute-Spatial Relation Alignment for 3D Visual Grounding
School of Information Science;Technology University of Science;School of Data Science University of Science;Department of Computer Science University of Rochester New York, United States
Flexible Tool Selection through Low-dimensional Attribute Alignment of Vision and Language
Laboratory of Brain Atlas and Brain-inspired Intelligence, Institute of Automation Chinese Academy of Sciences (CASIA)(中国科学院自动化研究所脑图谱与类脑智能实验室);School of Artificial Intelligence, University of Chinese Academy of Sciences (UCAS)(中国科学院大学人工智能学院);School of Systems Science, Beijing Normal University(北京师范大学系统科学学院);School of Psychological and Cognitive Sciences & Beijing Key Laboratory of Behavior and Mental Health, Peking University(北京大学心理与认知科学学院);IDG/McGovern Institute for Brain Research, Peking University(北京大学IDG/ McGovern脑科学研究院);Institute for Artificial Intelligence & Key Laboratory of Machine Perception (Ministry of Education), Peking University(北京大学人工智能研究所);School of Future Technology, University of Chinese Academy of Sciences (UCAS)(中国科学院大学未来技术学院)