C3L: Content Correlated Vision-Language Instruction Tuning Data Generation via Contrastive Learning
School of Computer Science, Northwestern Polytechnical University（西北工业大学计算机学院）；Ningbo Institute, Northwestern Polytechnical University（西北工业大学宁波研究院）；National Engineering Laboratory for Integrated Aero-Space-Ground-Ocean（空天地一体化综合业务网国家工程实验室）
Vision-Language Instruction Tuning (VLIT) is a critical training phase for Large Vision-Language Models (LVLMs). With the improving capabilities of open-source LVLMs, researchers have increasingly turned to generate VLIT data by using open-source LVLMs and achieved significant progress. However, such data generation approaches are bottlenecked by the following challenges: 1) Since multi-modal models tend to be influenced by prior language knowledge, directly using LVLMs to generate VLIT data would inevitably lead to low content relevance between generated data and images. 2) To improve the ability of the models to generate VLIT data, previous methods have incorporated an additional training phase to boost the generative capacity. This process hurts the generalization of the models to unseen inputs (i.e., "exposure bias" problem). In this paper, we propose a new Content Correlated VLIT data generation via Contrastive Learning (C3L). Specifically, we design a new content relevance module which enhances the content relevance between VLIT data and images by computing Image Instruction Correspondence Scores S(I2C). Moreover, a contrastive learning module is introduced to further boost the VLIT data generation capability of the LVLMs. A large number of automatic measures on four benchmarks show the effectiveness of our method.
VCM: Vision Concept Modeling Based on Implicit Contrastive Learning with Vision-Language Instruction Fine-Tuning
Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究所）；University of Chinese Academy of Sciences（中国科学院大学）；National University of Singapore（新加坡国立大学）；MoE Key Laboratory of Brain-inspired Intelligent Perception and Cognition, University of Science and Technology of China（中国科学技术大学脑启发智能感知与认知教育部重点实验室）
Weakly-supervised VLM-guided Partial Contrastive Learning for Visual Language Navigation
University of New South Wales（新南威尔士大学）；Adobe Research（Adobe研究）；University of California San Diego（加州大学圣地亚哥分校）；Northeastern University（东北大学）；CSIRO’s Data61 and University of New South Wales（CSIRO Data61和新南威尔士大学）
From Generalist to Specialist: Adapting Vision Language Models via Task-Specific Visual Instruction Tuning
Singapore Eye Research Institute, Singapore National Eye Centre（新加坡眼科研究所、新加坡国家眼科中心）；Duke-National University of Singapore Medical School（杜克-新加坡国立大学医学院）；Byers Eye Institute, Stanford University（斯坦福大学拜尔斯眼科研究所）
Instruction-Aligned Visual Attention for Mitigating Hallucinations in Large Vision-Language Models
School of Automation, Northwestern Polytechnical University（西北工业大学自动化学院）；School of Cybersecurity, Northwestern Polytechnical University（西北工业大学网络空间安全学院）；Alibaba Group（阿里巴巴集团）；Zhejiang University of Technology（浙江工业大学）；Binjiang Institute of Artificial Intelligence（滨江人工智能研究院）
Seeing the Image: Prioritizing Visual Correlation by Contrastive Alignment
School of Computer Science, Wuhan University（武汉大学计算机学院）；ByteDance Inc.（字节跳动公司）；School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）
Think Bright, Diffuse Nice: Enhancing T2I-ICL via Inductive-Bias Hint Instruction and Query Contrastive Decoding
Cao Tu Li (Guangzhou) Technology Co., Ltd（曹图利（广州）科技有限公司）；South China University of Technology（华南理工大学）；Washington State University（华盛顿州立大学）；Hong Kong Baptist University（香港 Baptist 大学）
HSCR: Hierarchical Self-Contrastive Rewarding for Aligning Medical Vision Language Models
Zhejiang University（浙江大学）；Byte Dance（字节跳动）；National University of Singapore（新加坡国立大学）；Angelalign Inc China（Angelalign中国公司）；Zhejiang Key Laboratory of Medical Imaging Artificial Intelligence（浙江省医学影像人工智能重点实验室）
LEARN: A Story-Driven Layout-to-Image Generation Framework for STEM Instruction
Graduate School of Advanced Science and Engineering, Hiroshima University（Hiroshima大学研究生院）；Department of Computer Science, Weifang University of Science and Technology（潍坊科技大学计算机科学系）
Semi-Supervised Video Desnowing Network via Temporal Decoupling Experts and Distribution-Driven Contrastive Regularization
The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；University of Cambridge（剑桥大学）；The Hong Kong University of Science and Technology（香港科技大学）
Contrastive Imitation Learning for Language-guided Multi-Task Robotic Manipulation
AI Thrust, The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州）AI Thrust）；The Hong Kong University of Science and Technology（香港科技大学）
Reproducible scaling laws for contrastive language-image learning
LAION；UC Berkeley（加州大学伯克利分校）；HuggingFace；University of Washington（华盛顿大学）；Juelich Supercomputing Center (JSC)（于利希超级计算中心）；Research Center Juelich (FZJ)（于利希研究中心）
Contrastive Conditional Latent Diffusion for Audio-visual Segmentation
School of Electronics and Information, Northwestern Polytechnical University（电子与信息学院，西北工业大学）；Shanghai AI Laboratory（上海人工智能实验室）；Shaanxi Key Laboratory of Information Acquisition and Processing（信息获取与处理陕西省重点实验室）
