多模态大模型

MMSRARec: Summarization and Retrieval Augumented Sequential Recommendation Based on Multimodal Large Language Model

MMSRARec: 基于多模态大语言模型的摘要与检索增强的序列推荐

Haoyu Wang, Yitong Wang, Jining Wang

专题命中 跨模态检索 ：multimodal(title,abstract)；MLLM(abstract)；image-text(abstract)；分类 cs.MM

AI总结 本文提出MMSRARec，基于多模态大语言模型，通过摘要与检索增强序列推荐，提升推荐性能、可解释性与计算效率。

Comments Under Review

Breaking the Modality Barrier: Universal Embedding Learning with Multimodal LLMs

突破模态壁垒：基于多模态大语言模型的通用嵌入学习

Tiancheng Gu, Kaicheng Yang, Ziyong Feng, Xingjun Wang, Yanzhao Zhang, Dingkun Long, Yingda Chen, Weidong Cai, Jiankang Deng

机构 * The University of Sydney（悉尼大学） ； DeepGlint ； Tongyi Lab, Alibaba Group（阿里云实验室） ； Imperial College London（伦敦帝国理工学院） 纠错

专题命中 跨模态检索 ：multimodal(title,abstract)；MLLM(abstract)；image-text(abstract)；分类 cs.CV

AI总结 UniME通过两阶段框架提升多模态表示学习，利用知识蒸馏和硬负样本微调增强判别能力与指令遵循性能。

Comments 13 pages, 8 figures, Accepted by ACM MM2025, Project page: https://garygutc.github.io/UniME

URaG: Unified Retrieval and Generation in Multimodal LLMs for Efficient Long Document Understanding

Yongxin Shi, Jiapeng Wang, Zeyu Shan, Dezhi Peng, Zening Lin, Lianwen Jin

专题命中 跨模态检索 ：multimodal(title,abstract)；MLLM(abstract)；cross-modal(abstract)；分类 cs.CL

Comments Accepted by AAAI 2026 (Oral)

ProM3E: Probabilistic Masked MultiModal Embedding Model for Ecology

Srikumar Sastry, Subash Khanal, Aayush Dhakal, Jiayu Lin, Dan Cher, Phoenix Jarosz, Nathan Jacobs

机构 * Washington University in St. Louis（华盛顿大学圣路易斯分校） 纠错

专题命中 跨模态检索 ：multimodal(title,abstract)；cross-modal(abstract)；any-to-any(abstract)；分类 cs.CV

Comments 21 pages, 16 figures

SAIL-Embedding Technical Report: Omni-modal Embedding Foundation Model

Lin Lin, Jiefeng Long, Zhihe Wan, Yuchi Wang, Dingkang Yang, Shuang Yang, Yueyang Yao, Xu Chen, Zirui Guo, Shengqiang Li, Weiran Li, Hanyu Li, Yaling Mou, Yan Qiu, Haiyang Yu, Xiao Liang, Hongsheng Li, Chao Feng

机构 * ByteDance Douyin SAIL Team, CUHK MMLab（字节跳动抖音SAIL团队，CUHK MMLab） 纠错

专题命中 跨模态检索 ：omni-modal(title,abstract)；multimodal(abstract)；cross-modal(abstract)；分类 cs.CV

Comments Technical Report

Exploring a Unified Vision-Centric Contrastive Alternatives on Multi-Modal Web Documents

Yiqi Lin, Alex Jinpeng Wang, Linjie Li, Zhengyuan Yang, Mike Zheng Shou

机构 * Show Lab, National University of Singapore（新加坡国立大学展示实验室） ； Central South University（中南大学） ； Microsoft（微软公司） 纠错

专题命中 跨模态检索 ：multi-modal(title)；multimodal(abstract)；cross-modal(abstract)；image-text(abstract)

Comments Project page: this https://linyq17.github.io/VC2L/

Generalized Contrastive Learning for Universal Multimodal Retrieval

Jungsoo Lee, Janghoon Cho, Hyojin Park, Munawar Hayat, Kyuwoong Hwang, Fatih Porikli, Sungha Choi

机构 * Qualcomm AI Research（高通人工智能研究） 纠错

专题命中 跨模态检索 ：multimodal(title,abstract)；cross-modal(abstract)；image-text(abstract)；分类 cs.CV

Comments Accepted to NeurIPS 2025

CPCL: Cross-Modal Prototypical Contrastive Learning for Weakly Supervised Text-based Person Retrieval

Xinpeng Zhao, Yanwei Zheng, Chuanlin Lan, Xiaowei Zhang, Bowen Huang, Jibin Yang, Dongxiao Yu

专题命中 跨模态检索 ：cross-modal(title,abstract)；multi-modal(abstract)；image-text(abstract)；分类 cs.CV

Comments 9 pages, 6 figures, under peer review

Towards Cross-modal Retrieval in Chinese Cultural Heritage Documents: Dataset and Solution

Junyi Yuan, Jian Zhang, Fangyu Wu, Dongming Lu, Huanda Lu, Qiufeng Wang

机构 * Xi’an Jiaotong-Liverpool University（西安交通大学利物浦大学） ； NingboTech University（宁波科技大学） ； Zhejiang University（浙江大学） 纠错

专题命中 跨模态检索 ：cross-modal(title,abstract)；multimodal(abstract)；image-text(abstract)；分类 cs.CV

Anatomy-Aware Conditional Image-Text Retrieval

Meng Zheng, Jiajin Zhang, Benjamin Planche, Zhongpai Gao, Terrence Chen, Ziyan Wu

专题命中 跨模态检索 ：image-text(title,abstract)；multimodal(abstract)；multi-modal(abstract)；分类 cs.CV

Comments 16 pages, 10 figures

Recurrence-Enhanced Vision-and-Language Transformers for Robust Multimodal Document Retrieval

Davide Caffagni, Sara Sarto, Marcella Cornia, Lorenzo Baraldi, Rita Cucchiara

专题命中 跨模态检索 ：multimodal(title,abstract)；cross-modal(abstract)；分类 cs.CV、cs.CL、cs.AI

Comments CVPR 2025

Explainable and Interpretable Multimodal Large Language Models: A Comprehensive Survey

Yunkai Dang, Kaichen Huang, Jiahao Huo, Yibo Yan, Sirui Huang, Dongrui Liu, Mengxi Gao, Jie Zhang, Chen Qian, Kun Wang, Yong Liu, Jing Shao, Hui Xiong, Xuming Hu

专题命中 跨模态检索 ：multimodal(title,abstract)；cross-modal(abstract)；image-text(abstract)；分类 cs.CL

Contrastive Knowledge Distillation for Robust Multimodal Sentiment Analysis

Zhongyi Sang, Kotaro Funakoshi, Manabu Okumura

专题命中 跨模态检索 ：multimodal(title,abstract)；multi-modal(abstract)；cross-modal(abstract)；分类 cs.MM

Multimodal Deep Learning for Low-Resource Settings: A Vector Embedding Alignment Approach for Healthcare Applications

David Restrepo, Chenwei Wu, Sebastián Andrés Cajas, Luis Filipe Nakayama, Leo Anthony Celi, Diego M López

专题命中 跨模态检索 ：multimodal(title,abstract)；multi-modal(abstract)；image-text(abstract)；分类 cs.AI

Multi-Modal Generative Embedding Model

Feipeng Ma, Hongwei Xue, Guangting Wang, Yizhou Zhou, Fengyun Rao, Shilin Yan, Yueyi Zhang, Siying Wu, Mike Zheng Shou, Xiaoyan Sun

专题命中 跨模态检索 ：multi-modal(title,abstract)；multimodal(abstract)；cross-modal(abstract)；分类 cs.CV

Hallucination Augmented Contrastive Learning for Multimodal Large Language Model

Chaoya Jiang, Haiyang Xu, Mengfan Dong, Jiaxing Chen, Wei Ye, Ming Yan, Qinghao Ye, Ji Zhang, Fei Huang, Shikun Zhang

专题命中 跨模态检索 ：multimodal(title)；multi-modal(abstract)；MLLM(abstract)；cross-modal(abstract)

Text-Guided Face Recognition using Multi-Granularity Cross-Modal Contrastive Learning

Md Mahedi Hasan, Shoaib Meraj Sami, Nasser Nasrabadi

专题命中 跨模态检索 ：cross-modal(title,abstract)；multimodal(abstract)；image-text(abstract)；分类 cs.CV

Comments Accepted at IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), 2024

Connecting Multi-modal Contrastive Representations

Zehan Wang, Yang Zhao, Xize Cheng, Haifeng Huang, Jiageng Liu, Li Tang, Linjun Li, Yongqi Wang, Aoxiong Yin, Ziang Zhang, Zhou Zhao

专题命中 跨模态检索 ：multi-modal(title,abstract)；audio-visual(abstract)；分类 cs.CV、cs.AI、cs.MM

Comments NeurIPS 2023

AdvCLIP: Downstream-agnostic Adversarial Examples in Multimodal Contrastive Learning

Ziqi Zhou, Shengshan Hu, Minghui Li, Hangtao Zhang, Yechao Zhang, Hai Jin

专题命中 跨模态检索 ：multimodal(title,abstract)；cross-modal(abstract)；image-text(abstract)；分类 cs.CV

Comments This paper has been accepted by the ACM International Conference on Multimedia (ACM MM '23, October 29-November 3, 2023, Ottawa, ON, Canada)

Using Multiple Instance Learning to Build Multimodal Representations

Peiqi Wang, William M. Wells, Seth Berkowitz, Steven Horng, Polina Golland

专题命中 跨模态检索 ：multimodal(title,abstract)；cross-modal(abstract)；image-text(abstract)；分类 cs.CV

RC3: Regularized Contrastive Cross-lingual Cross-modal Pre-training

Chulun Zhou, Yunlong Liang, Fandong Meng, Jinan Xu, Jinsong Su, Jie Zhou

专题命中 跨模态检索 ：cross-modal(title,abstract)；multi-modal(abstract)；image-text(abstract)；分类 cs.CL

Comments accepted to ACL 2023 Findings

Multimodal Federated Learning via Contrastive Representation Ensemble

Qiying Yu, Yang Liu, Yimu Wang, Ke Xu, Jingjing Liu

专题命中 跨模态检索 ：multimodal(title,abstract)；cross-modal(abstract)；image-text(abstract)；分类 cs.AI

Comments ICLR 2023, update

Learnable Pillar-based Re-ranking for Image-Text Retrieval

Leigang Qu, Meng Liu, Wenjie Wang, Zhedong Zheng, Liqiang Nie, Tat-Seng Chua

专题命中 跨模态检索 ：image-text(title,abstract)；multimodal(abstract)；cross-modal(abstract)；分类 cs.CV

Comments Accepted by SIGIR'2023

Journal ref Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 2023)

Cross-Modal Implicit Relation Reasoning and Aligning for Text-to-Image Person Retrieval

Ding Jiang, Mang Ye

专题命中 跨模态检索 ：cross-modal(title,abstract)；multimodal(abstract)；image-text(abstract)；分类 cs.CV

Comments Accepted by CVPR 2023. Codes are available at this https://github.com/anosorae/IRRA

ALADIN: Distilling Fine-grained Alignment Scores for Efficient Image-Text Matching and Retrieval

Nicola Messina, Matteo Stefanini, Marcella Cornia, Lorenzo Baraldi, Fabrizio Falchi, Giuseppe Amato, Rita Cucchiara

专题命中 跨模态检索 ：image-text(title,abstract)；cross-modal(abstract)；分类 cs.CV、cs.CL、cs.AI

Comments CBMI 2022

UNIMO: Towards Unified-Modal Understanding and Generation via Cross-Modal Contrastive Learning

Wei Li, Can Gao, Guocheng Niu, Xinyan Xiao, Hao Liu, Jiachen Liu, Hua Wu, Haifeng Wang

专题命中 跨模态检索 ：cross-modal(title,abstract)；multi-modal(abstract)；image-text(abstract)；分类 cs.CL

Comments The paper has been accepted by the main conference of ACL2021 as a long paper

Towards Efficient Cross-Modal Visual Textual Retrieval using Transformer-Encoder Deep Features

Nicola Messina, Giuseppe Amato, Fabrizio Falchi, Claudio Gennaro, Stéphane Marchand-Maillet

专题命中 跨模态检索 ：cross-modal(title,abstract)；multi-modal(abstract)；image-text(abstract)；分类 cs.CV

Comments Accepted at CBMI 2021

Learning Multi-Modal Nonlinear Embeddings: Performance Bounds and an Algorithm

Semih Kaya, Elif Vural

专题命中 跨模态检索 ：multi-modal(title,abstract)；cross-modal(abstract)；image-text(abstract)；分类 cs.CV

Continual learning in cross-modal retrieval

Kai Wang, Luis Herranz, Joost van de Weijer

专题命中 跨模态检索 ：cross-modal(title,abstract)；multimodal(abstract)；image-text(abstract)；分类 cs.CV

Comments 2nd CLVISION workshop in CVPR 2021

CHEF: Cross-modal Hierarchical Embeddings for Food Domain Retrieval

Hai X. Pham, Ricardo Guerrero, Jiatong Li, Vladimir Pavlovic

专题命中 跨模态检索 ：cross-modal(title,abstract)；multi-modal(abstract)；image-text(abstract)；分类 cs.CV

Comments 22 pages, accepted in AAAI 2021