多模态大模型

CustomContrast: A Multilevel Contrastive Perspective For Subject-Driven Text-to-Image Customization

Nan Chen, Mengqi Huang, Zhuowei Chen, Yang Zheng, Lei Zhang, Zhendong Mao

专题命中 跨模态检索 ：multimodal(abstract)；cross-modal(abstract)；分类 cs.CV、cs.MM

From Specific-MLLMs to Omni-MLLMs: A Survey on MLLMs Aligned with Multi-modalities

Shixin Jiang, Jiafeng Liang, Jiyuan Wang, Xuan Dong, Heng Chang, Weijiang Yu, Jinhua Du, Ming Liu, Bing Qin

专题命中 跨模态检索 ：multi-modal(abstract)；omni-modal(abstract)；分类 cs.CL、cs.AI

Comments 35 pages

Towards Text-Image Interleaved Retrieval

Xin Zhang, Ziqi Dai, Yongqi Li, Yanzhao Zhang, Dingkun Long, Pengjun Xie, Meishan Zhang, Jun Yu, Wenjie Li, Min Zhang

专题命中 跨模态检索 ：multimodal(abstract)；MLLM(abstract)；分类 cs.CV、cs.CL

Comments 16 pages, 14 figures

PerPO: Perceptual Preference Optimization via Discriminative Rewarding

Zining Zhu, Liang Zhao, Kangheng Lin, Jinze Yang, En Yu, Chenglong Liu, Haoran Wei, Jianjian Sun, Zheng Ge, Xiangyu Zhang

专题命中 跨模态检索 ：multimodal(abstract)；MLLM(abstract)；分类 cs.CL、cs.AI

SCOT: Self-Supervised Contrastive Pretraining For Zero-Shot Compositional Retrieval

Bhavin Jawade, Joao V. B. Soares, Kapil Thadani, Deen Dayal Mohan, Amir Erfan Eshratifar, Benjamin Culpepper, Paloma de Juan, Srirangaraj Setlur, Venu Govindaraju

专题命中 跨模态检索 ：multimodal(abstract)；image-text(abstract)；分类 cs.CV、cs.AI

Comments Paper accepted at WACV 2025 in round 1

No Captions, No Problem: Captionless 3D-CLIP Alignment with Hard Negatives via CLIP Knowledge and LLMs

Cristian Sbrolli, Matteo Matteucci

专题命中 跨模态检索 ：multimodal(abstract)；cross-modal(abstract)；分类 cs.CV、cs.AI

Comments to be published in BMVC 2024 Proceedings

Disentangled Noisy Correspondence Learning

Zhuohang Dang, Minnan Luo, Jihong Wang, Chengyou Jia, Haochen Han, Herun Wan, Guang Dai, Xiaojun Chang, Jingdong Wang

专题命中 跨模态检索 ：multi-modal(abstract)；cross-modal(abstract)；分类 cs.CV、cs.AI

Start from Video-Music Retrieval: An Inter-Intra Modal Loss for Cross Modal Retrieval

Zeyu Chen, Pengfei Zhang, Kai Ye, Wei Dong, Xin Feng, Yana Zhang

专题命中 跨模态检索 ：multi-modal(abstract)；cross-modal(abstract)；分类 cs.AI、cs.MM

Comments 10 pages, 7 figures

CaLa: Complementary Association Learning for Augmenting Composed Image Retrieval

Xintong Jiang, Yaxiong Wang, Mengjian Li, Yujiao Wu, Bingwen Hu, Xueming Qian

专题命中 跨模态检索 ：cross-modal(abstract)；image-text(abstract)；分类 cs.CV、cs.AI

Comments To appear at SIGIR 2024. arXiv admin note: text overlap with arXiv:2309.02169

Dual Relation Alignment for Composed Image Retrieval

Xintong Jiang, Yaxiong Wang, Yujiao Wu, Meng Wang, Xueming Qian

专题命中 跨模态检索 ：cross-modal(abstract)；image-text(abstract)；分类 cs.CV、cs.AI

Comments The architecture of our model changes, hence methodolgy and experiments changes a lot, We have significantly revised the original manuscript of the paper, so a withdraw of our original script is needed

Developing ChatGPT for Biology and Medicine: A Complete Review of Biomedical Question Answering

Qing Li, Lei Li, Yu Li

专题命中 跨模态检索 ：multimodal(abstract)；cross-modal(abstract)；分类 cs.CL、cs.AI

Comments 50 pages, 3 figures, 3 tables

Contrastive Learning-Based Spectral Knowledge Distillation for Multi-Modality and Missing Modality Scenarios in Semantic Segmentation

Aniruddh Sikdar, Jayant Teotia, Suresh Sundaram

专题命中 跨模态检索 ：multimodal(abstract)；multi-modal(abstract)；分类 cs.CV、cs.AI

Comments 10 pages, 6 figures

GrowCLIP: Data-aware Automatic Model Growing for Large-scale Contrastive Language-Image Pre-training

Xinchi Deng, Han Shi, Runhui Huang, Changlin Li, Hang Xu, Jianhua Han, James Kwok, Shen Zhao, Wei Zhang, Xiaodan Liang

专题命中 跨模态检索 ：cross-modal(abstract)；image-text(abstract)；分类 cs.CV、cs.AI

Comments Accepted by ICCV2023

Noisy Correspondence Learning with Meta Similarity Correction

Haochen Han, Kaiyao Miao, Qinghua Zheng, Minnan Luo

专题命中 跨模态检索 ：multimodal(abstract)；cross-modal(abstract)；分类 cs.CV、cs.MM

Comments Accepted at CVPR 2023

AToMiC: An Image/Text Retrieval Test Collection to Support Multimedia Content Creation

Jheng-Hong Yang, Carlos Lassance, Rafael Sampaio de Rezende, Krishna Srinivasan, Miriam Redi, Stéphane Clinchant, Jimmy Lin

专题命中 跨模态检索 ：cross-modal(abstract)；image-text(abstract)；分类 cs.CV、cs.CL

MedCLIP: Contrastive Learning from Unpaired Medical Images and Text

Zifeng Wang, Zhenbang Wu, Dinesh Agarwal, Jimeng Sun

专题命中 跨模态检索 ：multimodal(abstract)；image-text(abstract)；分类 cs.CV、cs.CL

Comments EMNLP 2022

3D-Augmented Contrastive Knowledge Distillation for Image-based Object Pose Estimation

Zhidan Liu, Zhen Xing, Xiangdong Zhou, Yijiang Chen, Guichun Zhou

专题命中 跨模态检索 ：multi-modal(abstract)；cross-modal(abstract)；分类 cs.CV、cs.AI

Comments Accepted for presentation at International Conference on Multimedia Retrieval (ICMR '22)

Large-scale Bilingual Language-Image Contrastive Learning

Byungsoo Ko, Geonmo Gu

专题命中 跨模态检索 ：multimodal(abstract)；image-text(abstract)；分类 cs.CV、cs.CL

Comments Accepted by ICLRW2022

Image Search with Text Feedback by Additive Attention Compositional Learning

Yuxin Tian, Shawn Newsam, Kofi Boakye

专题命中 跨模态检索 ：multi-modal(abstract)；image-text(abstract)；分类 cs.CV、cs.CL

SAC: Semantic Attention Composition for Text-Conditioned Image Retrieval

Surgan Jandial, Pinkesh Badjatiya, Pranit Chawla, Ayush Chopra, Mausoom Sarkar, Balaji Krishnamurthy

专题命中 跨模态检索 ：multi-modal(abstract)；image-text(abstract)；分类 cs.CV、cs.AI

Comments Surgan Jandial, Pinkesh Badjatiya, Pranit Chawla, and Ayush Chopra contributed equally to this work. Work accepted at WACV 2022

Contrastive Learning of Visual-Semantic Embeddings

Anurag Jain, Yashaswi Verma

专题命中 跨模态检索 ：multi-modal(abstract)；cross-modal(abstract)；分类 cs.CV、cs.MM

Contrastive Language-Image Pre-training for the Italian Language

Federico Bianchi, Giuseppe Attanasio, Raphael Pisoni, Silvia Terragni, Gabriele Sarti, Sri Lakshmi

专题命中 跨模态检索 ：multi-modal(abstract)；image-text(abstract)；分类 cs.CV、cs.CL

Video Corpus Moment Retrieval with Contrastive Learning

Hao Zhang, Aixin Sun, Wei Jing, Guoshun Nan, Liangli Zhen, Joey Tianyi Zhou, Rick Siow Mong Goh

专题命中 跨模态检索 ：multi-modal(abstract)；cross-modal(abstract)；分类 cs.CV、cs.CL

Comments 11 pages, 7 figures and 6 tables. Accepted by SIGIR 2021

T2VLAD: Global-Local Sequence Alignment for Text-Video Retrieval

Xiaohan Wang, Linchao Zhu, Yi Yang

专题命中 跨模态检索 ：multi-modal(abstract)；cross-modal(abstract)；分类 cs.CV、cs.MM

Comments Accepted to CVPR 2021

Learning to Represent Image and Text with Denotation Graph

Bowen Zhang, Hexiang Hu, Vihan Jain, Eugene Ie, Fei Sha

专题命中 跨模态检索 ：multimodal(abstract)；cross-modal(abstract)；分类 cs.CV、cs.CL

Comments to appear at EMNLP 2020

Probing Contextualized Sentence Representations with Visual Awareness

Zhuosheng Zhang, Rui Wang, Kehai Chen, Masao Utiyama, Eiichiro Sumita, Hai Zhao

专题命中 跨模态检索 ：multimodal(abstract)；cross-modal(abstract)；分类 cs.CV、cs.CL

Target-Oriented Deformation of Visual-Semantic Embedding Space

Takashi Matsubara

专题命中 跨模态检索 ：multimodal(abstract)；cross-modal(abstract)；分类 cs.CV、cs.MM

Comments 8 pages

VSE++: Improving Visual-Semantic Embeddings with Hard Negatives

Fartash Faghri, David J. Fleet, Jamie Ryan Kiros, Sanja Fidler

专题命中 跨模态检索 ：multi-modal(abstract)；cross-modal(abstract)；分类 cs.CV、cs.CL

Comments Accepted as spotlight presentation at British Machine Vision Conference (BMVC) 2018. Code: https://github.com/fartashf/vsepp

Finding beans in burgers: Deep semantic-visual embedding with localization

Martin Engilberge, Louis Chevallier, Patrick Pérez, Matthieu Cord

专题命中 跨模态检索 ：multi-modal(abstract)；cross-modal(abstract)；分类 cs.CV、cs.CL

Comments Accepted to CVPR2018

CMRAG: Co-modality-based visual document retrieval and question answering

CMRAG：基于共模的视觉文档检索与问答

Wang Chen, Wenhan Yu, Guanqiang Qi, Weikang Li, Yang Li, Lei Sha, Deguo Xia, Jizhou Huang

机构 * Baidu Inc（百度公司） ； The University of Hong Kong（香港大学） ； Beihang University（北京航空航天大学） ； Peking University（北京大学） 纠错

专题命中 跨模态检索 ：multimodal(abstract,comments)；cross-modal(abstract)；分类 cs.CL

AI总结 CMRAG通过统一编码模型和共模信息检索方法，提升多模态视觉文档问答系统的性能。

Comments Published at ICLR 2026 Workshop on Multimodal Intelligence