Multi-modal pre-training models have been intensively explored to bridge vision and language in recent years. However, most of them explicitly model the cross-modal interaction between image-text pairs, by assuming that there exists strong semantic correlation between the text and image modalities. Since this strong assumption is often invalid in real-world scenarios, we choose to implicitly model the cross-modal correlation for large-scale multi-modal pre-training, which is the focus of the Chinese project `WenLan' led by our team. Specifically, with the weak correlation assumption over image-text pairs, we propose a two-tower pre-training model called BriVL within the cross-modal contrastive learning framework. Unlike OpenAI CLIP that adopts a simple contrastive learning method, we devise a more advanced algorithm by adapting the latest method MoCo into the cross-modal scenario. By building a large queue-based dictionary, our BriVL can incorporate more negative samples in limited GPU resources. We further construct a large Chinese multi-source image-text dataset called RUC-CAS-WenLan for pre-training our BriVL model. Extensive experiments demonstrate that the pre-trained BriVL model outperforms both UNITER and OpenAI CLIP on various downstream tasks.
MCEN: Bridging Cross-Modal Gap between Cooking Recipes and Dish Images with Latent Variable Model
Zhejiang University（浙江大学）；Singapore Management University（新加坡管理大学）；Alibaba-Zhejiang University Joint Institute of Frontier Technologies（阿里巴巴-浙江大学前沿技术联合研究院）
CM-Bench: A Comprehensive Cross-Modal Feature Matching Benchmark Bridging Visible and Infrared Images
School of Instrument Science and Opto-Electronics Engineering, Beijing Information Science and Technology University（北京信息科技大学仪器科学与光电工程学院）；Department of Precision Instrument, Tsinghua University（清华大学精密仪器系）
Representation Discrepancy Bridging Method for Remote Sensing Image-Text Retrieval
School of Computer Science and Technology, Xi’an University of Posts and Telecommunications（西安邮电大学计算机科学与技术学院）；Shaanxi Key Laboratory of Network Data Analysis and Intelligent Processing（陕西省网络数据分析与智能处理重点实验室）；Xi’an Key Laboratory of Big Data and Intelligent Computing（西安大数据与智能计算重点实验室）；School of Computer Engineering, Weifang University（潍坊大学计算机工程学院）；Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）；Department of Electronic and Electrical Engineering, Brunel University of London（伦敦布鲁内尔大学电子与电气工程系）；Dept. Matemàtiques i Informàtica and Institute of Neuroscience, Univeritat de Barcelona（数学与计算机系和神经科学研究所，巴塞罗那大学）
ChatBridge: Bridging Modalities with Large Language Model as a Language Catalyst
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）；Bytedance Inc.（字节跳动公司）
Bridging Ears and Eyes: Analyzing Audio and Visual Large Language Models to Humans in Visible Sound Recognition and Reducing Their Sensory Gap via Cross-Modal Distillation
Columbia University（哥伦比亚大学）；University of Washington（华盛顿大学）
DetectiumFire: A Comprehensive Multi-modal Dataset Bridging Vision and Language for Fire Understanding
Department of Computer Science（计算机科学系）；Tulane University（Tulane 大学）；Department of Industrial Engineering and Management（工业工程与管理系）；Aalto University（Aalto 大学）
PivotMerge: Bridging Heterogeneous Multimodal Pre-training via Post-Alignment Model Merging
State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences（多模态人工智能系统国家重点实验室，中国科学院自动化研究所）；Pengcheng Laboratory（鹏城实验室）；School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）；Data Science and Artificial Intelligence Research Institute, China United Network Communications Group Co., Ltd.（中国联合网络通信集团有限公司数据科学与人工智能研究院）
TOLiD: Bridging the Architecture Gap in Vision Foundation Model to LiDAR Pretraining via Token Lifting for Distillation
SAIVT Group, School of Electrical Engineering and Robotics, Queensland University of Technology（澳大利亚昆士兰科技大学电气工程与机器人学院SAIVT组）；CSIRO Robotics, CSIRO（澳大利亚联邦科学与工业研究组织机器人部）
Cross-Modality Gait Recognition: Bridging LiDAR and Camera Modalities for Human Identification
Southern University of Science and Technology（南方科技大学）；The University of Hong Kong（香港大学）；University of Cordoba（科尔多瓦大学）；The Hong Kong Polytechnic University（香港理工大学）
KG-ViP: Bridging Knowledge Grounding and Visual Perception in Multi-modal LLMs for Visual Question Answering
University of Science and Technology of China（中国科学技术大学）；Data Darkness Lab, MIRACLE Center, USTC（数据黑暗实验室，MIRACLE中心，中国科学技术大学）；School of Computer Science and Technology, Xidian University（西安电子科技大学计算机科学与技术学院）
Bridging Modal Isolation in Interleaved Thinking: Supervising Modality Transitions via Stepwise Reinforcement
Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）；Shanghai Jiaotong University（上海交通大学）；Zhejiang University（浙江大学）；University of Chinese Academy of Sciences（中国科学院大学）
Hg-I2P: Bridging Modalities for Generalizable Image-to-Point-Cloud Registration via Heterogeneous Graphs
Huazhong University of Science and Technology（华中科技大学）；Northwestern Polytechnical University（西北工业大学）；Huazhong Agricultural University（华中农业大学）；Delft University of Technology（代尔夫特理工大学）
Bridging Severe Cross-Modal Misalignment: End-to-End Visible-Infrared Object Detection via Explicit Feature-Domain Affine Registration
Beijing University of Technology（北京工业大学）；Central South University（中南大学）；Beijing Electronics Science & Technology Institute（北京电子科学技术研究所）；China Aerospace Science & Industry Corporation（中国航天科技集团）；Beijing Institute of Technology（北京理工大学）；Information Support Force Engineering University（信息支援部队工程大学）；Trunk Technology (Beijing) Co., Ltd.（trunk技术（北京）有限公司）
Self-Supervised Enhancement of Forward-Looking Sonar Images: Bridging Cross-Modal Degradation Gaps through Feature Space Transformation and Multi-Frame Fusion
College of Electronic Engineering, Naval University of Engineering（电子工程学院，海军工程大学）；College of Meteorology and Oceanography, National University of Defense Technology（气象海洋学院，国防科技大学）；College of Computer Science and Technology, National University of Defense Technology（计算机科学与技术学院，国防科技大学）；Department of Computer Science and Technology, Tsinghua University（计算机科学与技术系，清华大学）
Bridging Modalities and Tasks: A Unified Hierarchical ViT for SAR-to-Optical Translation and Semantic Segmentation
School of Automation, Northwestern Polytechnical University（西北工业大学自动化学院）；School of Cybersecurity, Northwestern Polytechnical University（西北工业大学网络空间安全学院）
Category-level Text-to-Image Retrieval Improved: Bridging the Domain Gap with Diffusion Models and Vision Encoders
King Abdullah University of Science and Technology（国王阿卜杜勒-阿齐兹大学科学与技术大学）；Visual Recognition Group（视觉识别组）；Faculty of Electrical Engineering（电气工程学院）；Czech Technical University in Prague（布拉格捷克技术大学）
