Vision-Language Pre-training (VLP) has advanced the performance for many vision-language tasks. However, most existing pre-trained models only excel in either understanding-based tasks or generation-based tasks. Furthermore, performance improvement has been largely achieved by scaling up the dataset with noisy image-text pairs collected from the web, which is a suboptimal source of supervision. In this paper, we propose BLIP, a new VLP framework which transfers flexibly to both vision-language understanding and generation tasks. BLIP effectively utilizes the noisy web data by bootstrapping the captions, where a captioner generates synthetic captions and a filter removes the noisy ones. We achieve state-of-the-art results on a wide range of vision-language tasks, such as image-text retrieval (+2.7% in average recall@1), image captioning (+2.8% in CIDEr), and VQA (+1.6% in VQA score). BLIP also demonstrates strong generalization ability when directly transferred to video-language tasks in a zero-shot manner. Code, models, and datasets are released at https://github.com/salesforce/BLIP.
Dynamic Multimodal Evaluation with Flexible Complexity by Vision-Language Bootstrapping
School of Artificial Intelligence, Shanghai Jiao Tong University（上海交通大学人工智能学院）；Shanghai AI Laboratory（上海人工智能实验室）；Tongji University（同济大学）；The University of Hong Kong（香港大学）
Bootstrapping Vision-language Models for Self-supervised Remote Physiological Measurement
College of Electronic and Information Engineering, Tongji University（同济大学电子与信息工程学院）；Shanghai Institute of Intelligent Science and Technology, Tongji University（同济大学上海智能科学与技术研究院）；School of Management, Hefei University of Technology（合肥工业大学管理学院）
Abn-BLIP: Abnormality-aligned Bootstrapping Language-Image Pre-training for Pulmonary Embolism Diagnosis and Report Generation from CTPA
Department of Diagnostic Imaging, Brown University Health（布朗大学健康中心诊断影像科）；Warren Alpert Medical School of Brown University（布朗大学沃伦·阿尔珀特医学院）；Department of Biomedical Engineering, Johns Hopkins University School of Medicine（约翰霍普金斯大学医学院生物医学工程系）；Department of Radiology and Radiological Sciences, Johns Hopkins University School of Medicine（约翰霍普金斯大学医学院放射科）；Johns Hopkins University Division of Pulmonary and Critical Care Medicine（约翰霍普金斯大学肺科与重症医学科）；Department of Radiology, University of Colorado School of Medicine（科罗拉多大学医学院放射科）
Learning by Correction: Efficient Tuning Task for Zero-Shot Generative Vision-Language Reasoning
School of Information Science and Technology, ShanghaiTech University（上海科技大学信息科学与技术学院）；Shanghai Engineering Research Center of Intelligent Vision and Imaging（上海智能视觉与成像工程技术研究中心）
Revisiting CroPA: A Reproducibility Study and Enhancements for Cross-Prompt Adversarial Transferability in Vision-Language Models
Mehta Family School of Data Science and Artificial Intelligence（梅hta家族数据科学与人工智能学院）；Indian Institute of Technology, Roorkee（印度理工学院罗奥克学院）；Department of Civil Engineering（土木工程系）；Department of Electronics and Communication（电子与通信系）
On Evaluating Adversarial Robustness of Large Vision-Language Models
Tsinghua University（清华大学）；Renmin University of China（中国人民大学）；Sea AI Lab, Singapore（新加坡Sea AI实验室）；Singapore University of Technology and Design（新加坡科技设计大学）
University of Arkansas（阿肯色大学）；University of Science, VNU-HCM（胡志明市国家大学科学大学）；Aalto University（阿尔托大学）；SpexAI GmbH（SpexAI有限公司）；University of California, San Diego（加利福尼亚大学圣迭戈分校）
Revisiting Backdoor Attacks against Large Vision-Language Models from Domain Shift
National University of Singapore（新加坡国立大学）；Shenzhen Campus of Sun Yat-sen University（中山大学深圳校区）；Sea AI lab（海天人工智能实验室）；The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））；Nanyang Technological University（南洋理工大学）
Closed-Loop Bidirectional Prompting for Adversarial Robustness of Vision Language Models
University of Macau（澳门大学）；Guangdong Institute of Intelligence Science and Technology（广东智能科学与技术研究院）；Peking University（北京大学）；Independent Researcher（独立研究员）；Institute of Science Tokyo（东京科学研究院）；Morgan Stanley（摩根大通）；Halmstad University（哈马碧大学）
Bootstrapping MLLM for Weakly-Supervised Class-Agnostic Object Counting
College of Electronic and Information Engineering, Tongji University（电子信息工程学院，同济大学）；College of Computer Science, Wuhan University（计算机科学学院，武汉大学）；College of Computer Science, Tongji University（计算机科学学院，同济大学）；State Key Laboratory of Autonomous Intelligent Unmanned Systems（自主智能无人系统国家重点实验室）
Conceptualizing Embeddings: Sparse Disentanglement for Vision-Language Models
Faculty of Mathematics and Computer Science, Jagiellonian University（雅盖隆大学数学与计算机科学学院）；Doctoral School of Exact and Natural Sciences, Jagiellonian University（雅盖隆大学精确与自然科学博士学校）；Centre for Credible AI, Warsaw University of Technology（华沙技术大学可信人工智能中心）
Are Neuro-Inspired Multi-Modal Vision-Language Models Resilient to Membership Inference Privacy Leakage?
The University of Alabama（阿拉巴马大学）；Alabama Center for the Advancement of AI（阿拉巴马人工智能 advancement 中心）；Trustworthy AI Lab（可信人工智能实验室）；Department of Computer Science, The University of Alabama（计算机科学系）
