Attribute labeling at large scale is typically incomplete and partial, posing significant challenges to model optimization. Existing attribute learning methods often treat the missing labels as negative or simply ignore them all during training, either of which could hamper the model performance to a great extent. To overcome these limitations, in this paper we leverage the available vision-language knowledge to explicitly disclose the missing labels for enhancing model learning. Given an image, we predict the likelihood of each missing attribute label assisted by an off-the-shelf vision-language model, and randomly select to ignore those with high scores in training. Our strategy strikes a good balance between fully ignoring and negatifying the missing labels, as these high scores are found to be informative on revealing label ambiguity. Extensive experiments show that our proposed vision-language assisted loss can achieve state-of-the-art performance on the newly cleaned VAW dataset. Qualitative evaluation demonstrates the ability of the proposed method in predicting more complete attributes.
CASA: Class-Agnostic Shared Attributes in Vision-Language Models for Efficient Incremental Object Detection
School of Electronic and Computer Engineering, Peking University(北京大学电子与计算机工程学院);Peng Cheng Laboratory(鹏城实验室);School of Computer Science, Peking University(北京大学计算机学院)
Vision-to-Language Tasks Based on Attributes and Attention Mechanism
Northwestern Polytechnical University(西北工业大学);Xi’an Institute of Optics and Precision Mechanics, Chinese Academy of Sciences(中国科学院西安光学精密机械研究所);University of Chinese Academy of Sciences(中国科学院大学)
Toward Robust Medical Fairness: Debiased Dual-Modal Alignment via Text-Guided Attribute-Disentangled Prompt Learning for Vision-Language Models
National Engineering Laboratory for Integrated Aero-Space-Ground-Ocean Big Data Application Technology(集成空天地海大数据应用技术国家工程实验室);Northwestern Polytechnical University(西北工业大学);Huiying Medical Technology Company Ltd.(慧影医疗技术有限公司);The School of Computer Science(计算机学院);The University of Sydney(悉尼大学);Department of Computer Science and Engineering(计算机科学与工程系);The Chinese University of Hong Kong(香港中文大学)
The Eye of Sherlock Holmes: Uncovering User Private Attribute Profiling via Vision-Language Model Agentic Framework
Nanyang Technological University(南洋理工大学);Beijing University of Technology(北京理工大学);Hengxin Tech(恒心科技);Alibaba Group(阿里巴巴集团);Squirrel Ai Learning(squirrel AI 学习)
SemPT: Semantic Prompt Tuning for Vision-Language Models
School of Computer Science and Artificial Intelligence, Wuhan Textile University(武汉纺织大学计算机科学与人工智能学院);School of Remote Sensing and Information Engineering, Wuhan University(武汉大学遥感与信息工程学院)
VLM-PAR: A Vision Language Model for Pedestrian Attribute Recognition
Department of Innovation Engineering(创新工程系);University of Salento, Italy(意大利萨伦托大学);Institute of Applied Sciences and Intelligent Systems - CNR(应用科学与智能系统研究所 - CNR);University of the Basque Country UPV/EHU(巴斯克国家大学UPV/EHU);IKERBASQUE, Basque Foundation for Science(伊基塔斯克巴塞克基金会);Sorbonne University Abu Dhabi(索邦大学阿布扎比分校)
Causality-guided Prompt Learning for Vision-language Models via Visual Granulation
State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences(多模态人工智能系统国家重点实验室,自动化研究所,中国科学院);School of Artificial Intelligence, University of Chinese Academy of Sciences(人工智能学院,中国科学院大学)
Pedestrian Attribute Recognition via CLIP based Prompt Vision-Language Fusion
School of Computer Science and Technology, Anhui University(安徽大学计算机科学与技术学院);Institute of Artificial Intelligence, Hefei Comprehensive National Science Center(合肥综合性国家科学中心人工智能研究院);Information Materials and Intelligent Sensing Laboratory of Anhui Province(安徽省信息材料与智能感知实验室);Anhui Provincial Key Laboratory of Multimodal Cognitive Computation(安徽省多模态认知计算重点实验室);School of Artificial Intelligence, Anhui University(安徽大学人工智能学院);Video Investigation Detachment of Hefei Public Security Bureau(合肥市公安局视频侦查支队)
Diagnosing Dense Same-Class Attribute Misbinding in Large Vision-Language Models
Qilu University of Technology (Shandong Academy of Sciences)(齐鲁工业大学(山东省科学院));China Telecom Digital Intelligence Technology Co., Ltd.(中国电信数字智能科技有限公司);Shenyang Aerospace University(沈阳航空航天大学);University of Nottingham Ningbo China(宁波诺丁汉大学)
Dual Prompt Learning for Adapting Vision-Language Models to Downstream Image-Text Retrieval
College of Computer Science, Sichuan University(四川大学计算机学院);Engineering Research Center of Machine Learning and Industry Intelligence, Ministry of Education, Chengdu, China(教育部机器学习与产业智能工程研究中心);Deep NeuroCognition Lab, I2R and CFAR, Agency for Science, Technology and Research, Singapore(深度神经认知实验室,I2R和CFAR,科技研究局,新加坡)
SurgiSR4K: A High-Resolution Endoscopic Video Dataset for Robotic-Assisted Minimally Invasive Procedures
Intuitive Surgical, Inc.(Intuitive Surgical公司);Johns Hopkins Medicine Neurosurgery(约翰霍普金斯医学神经外科);Johns Hopkins University Electrical and Computer Engineering(约翰霍普金斯大学电气与计算机工程);University of British Columbia Electrical and Computer Engineering(不列颠哥伦比亚大学电气与计算机工程);Wilford & Kate Bailey Small Animal Teaching Hospital(威尔福德与凯蒂·贝利小动物教学医院)
Social Perception of Faces in a Vision-Language Model
California Institute of Technology(加州理工学院);ETH Zurich(苏黎世联邦理工学院);Computational Social Science(计算社会科学);Swiss Data Science Center(瑞士数据科学中心);Empa(瑞士联邦材料科学与技术研究院)