多模态大模型

SDR-CIR: Semantic Debias Retrieval Framework for Training-Free Zero-Shot Composed Image Retrieval

SDR-CIR：一种基于链式推理的语义去偏检索框架用于无训练零样本复合图像检索

Yi Sun, Jinyu Xu, Qing Xie, Jiachen Li, Yanchun Ma, Yongjian Liu

专题命中 跨模态检索 ：multimodal(abstract)；MLLM(abstract)

AI总结 SDR-CIR通过语义去偏排名方法提升无训练零样本复合图像检索性能。

Comments Accepted by WWW 2026

GMS-CAVP: Improving Audio-Video Correspondence with Multi-Scale Contrastive and Generative Pretraining

GMS-CAVP：通过多尺度对比和生成预训练提升音频视频对应关系

Shentong Mo, Zehua Chen, Jun Zhu

机构 * Carnegie Mellon University（卡内基梅隆大学） ； MBZUAI ； Tsinghua University（清华大学） 纠错

专题命中 跨模态检索 ：cross-modal(abstract)；分类 cs.CV、cs.AI、eess.AS

AI总结 GMS-CAVP通过多尺度对比和生成预训练方法提升音频视频对应关系建模，实现更深入的跨模态理解和高保真生成。

Towards Safety-Compliant Transformer Architectures for Automotive Systems

面向汽车系统的安全合规Transformer架构

Sven Kirchner, Nils Purschke, Chengdong Wu, Alois Knoll

专题命中 跨模态检索 ：multimodal(abstract)；multimodal foundation model(abstract)

AI总结 本文提出了一种安全合规的Transformer架构，通过多模态基础模型提升汽车系统的容错性和鲁棒性，实现自动驾驶中的可认证AI系统。

SeisBind: Physics-Aware Tri-Modal Representation Binding for Seismic Data via Contrastive Learning

SeisBind：通过对比学习实现地震数据的物理感知三模态表示绑定

Chaohua Liang, Jun Matsushima

专题命中 跨模态检索 ：multi-modal(abstract)；cross-modal(abstract)

AI总结 SeisBind通过对比学习将地震数据与物理描述符结合，实现更可解释的地下速度模型构建。

Retrieval-augmented Prompt Learning for Pre-trained Foundation Models

基于检索的提示学习用于预训练基础模型

Xiang Chen, Yixin Ou, Quan Feng, Lei Li, Piji Li, Haibo Ye, Sheng-Jun Huang, Shuofei Qiao, Shumin Deng, Huajun Chen, Ningyu Zhang

机构 * MIIT Key Laboratory of Pattern Analysis and Machine Intelligence, College of Computer Science and Technology, Nanjing University of Aeronautics and Astronautics（信息产业部模式分析与机器智能重点实验室，计算机科学与技术学院，南京航空航天大学） ； Zhejiang University（浙江大学） ； Hunan Vanguard Group Corporation Limited（湖南先锋集团有限公司） ； National University of Singapore（新加坡国立大学） 纠错

专题命中 跨模态检索 ：multimodal(abstract)；分类 cs.CV、cs.CL、cs.AI

AI总结 RetroPrompt通过整合检索机制和知识库，提升预训练基础模型在少样本和零样本场景下的泛化能力与记忆平衡。

Comments IEEE/ACM Transactions on Audio, Speech and Language Processing

CLIBD: Bridging Vision and Genomics for Biodiversity Monitoring at Scale

CLIBD：融合视觉与基因组学用于大规模生物多样性监测

ZeMing Gong, Austin T. Wang, Xiaoliang Huo, Joakim Bruslund Haurum, Scott C. Lowe, Graham W. Taylor, Angel X. Chang

机构 * Simon Fraser University（西蒙弗雷泽大学） ； Aalborg University（奥胡斯大学） ； Vector Institute（向量研究所） ； University of Guelph（圭尔夫大学） ； Alberta Machine Intelligence Institute (Amii)（阿尔伯塔人工智能研究所） 纠错

专题命中 跨模态检索 ：multimodal(abstract)；分类 cs.CV、cs.CL、cs.AI

AI总结 CLIBD通过融合视觉与基因组学数据，利用对比学习实现对昆虫物种的高效分类，提升生物多样性监测的准确性。

Comments Add Variations of DNA encoding

Unsupervised Memorability Modeling from Tip-of-the-Tongue Retrieval Queries

从舌尖上的遗忘检索查询中无监督建模可记忆性

Sree Bhattacharyya, Yaman Kumar Singla, Sudhir Yarram, Somesh Kumar Singh, Harini S, James Z. Wang

机构 * The Pennsylvania State University（宾夕法尼亚州立大学） ； Adobe Media and Data Science Research（Adobe媒体与数据科学研究） 纠错

专题命中 跨模态检索 ：multimodal(abstract)；分类 cs.CV、cs.CL、cs.AI

AI总结 本文提出首个大规模无监督数据集，通过舌尖上的遗忘检索查询建模视觉内容的可记忆性，并展示了其在回忆生成和ToT检索任务中的优越表现。

Comments Accepted at WACV 2026

EBind: a practical approach to space binding

Jim Broadbent, Felix Cohen, Frederik Hvilshøj, Eric Landau, Eren Sasoglu

机构 * Encord London, UK（Encord伦敦） 纠错

专题命中 跨模态检索 ：multimodal(abstract)；image-text(abstract)

BenCao: An Instruction-Tuned Large Language Model for Traditional Chinese Medicine

Jiacheng Xie, Yang Yu, Yibo Chen, Hanyao Zhang, Lening Zhao, Jiaxuan He, Lei Jiang, Xiaoting Tang, Guanghui An, Dong Xu

机构 * Community Health Service Center Shanghai Pudong New Area（上海浦东新区社区卫生服务中心） ； School of Acupuncture-Moxibustion and Tuina, Shanghai University of Traditional Chinese Medicine（上海中医药大学针灸推拿学院） 纠错

专题命中 跨模态检索 ：multimodal(abstract)；分类 cs.CL、cs.AI、cs.MM

Thin Bridges for Drug Text Alignment: Lightweight Contrastive Learning for Target Specific Drug Retrieval

Mallikarjuna Tupakula

机构 * Rochester Institute of Technology（罗切斯特技术研究所） 纠错

专题命中 跨模态检索 ：multimodal(abstract)；multimodal foundation model(abstract)

Closing the Modality Gap for Mixed Modality Search

Binxu Li, Yuhui Zhang, Xiaohan Wang, Weixin Liang, Ludwig Schmidt, Serena Yeung-Levy

机构 * Stanford University（斯坦福大学） 纠错

专题命中 跨模态检索 ：multimodal(abstract)；分类 cs.CV、cs.CL、cs.AI

Comments Project page: https://yuhui-zh15.github.io/MixedModalitySearch/

From Pixels to CSI: Distilling Latent Dynamics For Efficient Wireless Resource Management

Charbel Bou Chaaya, Abanoub M. Girgis, Mehdi Bennis

专题命中 跨模态检索 ：multimodal(abstract)；cross-modal(abstract)

ViDoRAG: Visual Document Retrieval-Augmented Generation via Dynamic Iterative Reasoning Agents

Qiuchen Wang, Ruixue Ding, Zehui Chen, Weiqi Wu, Shihang Wang, Pengjun Xie, Feng Zhao

机构 * MoE Key Laboratory of Brain-inspired Intelligent Perception and Cognition, USTC（脑启发智能感知与认知联合实验室，中国科学技术大学） ； Tongyi Lab, Alibaba Group（通义实验室，阿里巴巴集团） ； Shanghai Jiao Tong University（上海交通大学） 纠错

专题命中 跨模态检索 ：multi-modal(abstract)；分类 cs.CV、cs.CL、cs.AI

From ML to LLM: Evaluating the Robustness of Phishing Webpage Detection Models against Adversarial Attacks

Aditya Kulkarni, Vivek Balachandran, Dinil Mon Divakaran, Tamal Das

专题命中 跨模态检索 ：multimodal(abstract)；MLLM(abstract)

CAFe: Unifying Representation and Generation with Contrastive-Autoregressive Finetuning

Hao Yu, Zhuokai Zhao, Shen Yan, Lukasz Korycki, Jianyu Wang, Baosheng He, Jiayi Liu, Lizhu Zhang, Xiangjun Fan, Hanchao Yu

专题命中 跨模态检索 ：multimodal(abstract)；分类 cs.CV、cs.CL、cs.AI

A Comprehensive Survey on Composed Image Retrieval

Xuemeng Song, Haoqiang Lin, Haokun Wen, Bohan Hou, Mingzhu Xu, Liqiang Nie

专题命中 跨模态检索 ：multimodal(abstract)；分类 cs.CV、cs.AI、cs.MM

Contrastive Language Prompting to Ease False Positives in Medical Anomaly Detection

YeongHyeon Park, Myung Jin Kim, Hyeong Seok Kim

专题命中 跨模态检索 ：multi-modal(abstract)；分类 cs.CV、cs.CL、cs.AI

Comments 4 pages, 3 figures, 2 tables

Text-guided Image Restoration and Semantic Enhancement for Text-to-Image Person Retrieval

Delong Liu, Haiwen Li, Zhicheng Zhao, Yuan Dong

专题命中 跨模态检索 ：cross-modal(abstract)；分类 cs.CV、cs.CL、cs.AI

Comments The paper was withdrawn due to a dispute among the authors regarding the content of the article

Enhancing CLIP Conceptual Embedding through Knowledge Distillation

Kuei-Chun Kao

专题命中 跨模态检索 ：multi-modal(abstract)；分类 cs.CV、cs.CL、cs.AI

GeomCLIP: Contrastive Geometry-Text Pre-training for Molecules

Teng Xiao, Chao Cui, Huaisheng Zhu, Vasant G. Honavar

专题命中 跨模态检索 ：multimodal(abstract)；multi-modal(abstract)

Comments BIBM 2024

INQUIRE: A Natural World Text-to-Image Retrieval Benchmark

Edward Vendrow, Omiros Pantazis, Alexander Shepard, Gabriel Brostow, Kate E. Jones, Oisin Mac Aodha, Sara Beery, Grant Van Horn

专题命中 跨模态检索 ：multimodal(abstract)；分类 cs.CV、cs.CL、cs.AI

Comments Published in NeurIPS 2024, Datasets and Benchmarks Track

TaxaBind: A Unified Embedding Space for Ecological Applications

Srikumar Sastry, Subash Khanal, Aayush Dhakal, Adeel Ahmad, Nathan Jacobs

专题命中 跨模态检索 ：multimodal(abstract)；分类 cs.CV、cs.CL、cs.AI

Comments Accepted to WACV 2025

SCOPE: Sign Language Contextual Processing with Embedding from LLMs

Yuqi Liu, Wenqian Zhang, Sihan Ren, Chengyu Huang, Jingyi Yu, Lan Xu

专题命中 跨模态检索 ：multi-modal(abstract)；分类 cs.CV、cs.CL、cs.AI

Revolutionizing Text-to-Image Retrieval as Autoregressive Token-to-Voken Generation

Yongqi Li, Hongru Cai, Wenjie Wang, Leigang Qu, Yinwei Wei, Wenjie Li, Liqiang Nie, Tat-Seng Chua

专题命中 跨模态检索 ：cross-modal(abstract)；分类 cs.CV、cs.AI、cs.MM

Comments Work in progress

Interfacing Foundation Models' Embeddings

Xueyan Zou, Linjie Li, Jianfeng Wang, Jianwei Yang, Mingyu Ding, Junyi Wei, Zhengyuan Yang, Feng Li, Hao Zhang, Shilong Liu, Arul Aravinthan, Yong Jae Lee, Lijuan Wang

专题命中 跨模态检索 ：multi-modal(abstract)；分类 cs.CV、cs.CL、cs.AI

Comments CODE: https://github.com/UX-Decoder/FIND

MATE: Meet At The Embedding -- Connecting Images with Long Texts

Young Kyun Jang, Junmo Kang, Yong Jae Lee, Donghyun Kim

专题命中 跨模态检索 ：cross-modal(abstract)；分类 cs.CV、cs.CL、cs.AI

ComCLIP: Training-Free Compositional Image and Text Matching

Kenan Jiang, Xuehai He, Ruize Xu, Xin Eric Wang

专题命中 跨模态检索 ：image-text(abstract)；分类 cs.CV、cs.CL、cs.AI

Retrieval-based Disentangled Representation Learning with Natural Language Supervision

Jiawei Zhou, Xiaoguang Li, Lifeng Shang, Xin Jiang, Qun Liu, Lei Chen

专题命中 跨模态检索 ：cross-modal(abstract)；分类 cs.CV、cs.CL、cs.AI

BioBridge: Bridging Biomedical Foundation Models via Knowledge Graphs

Zifeng Wang, Zichen Wang, Balasubramaniam Srinivasan, Vassilis N. Ioannidis, Huzefa Rangwala, Rishita Anubhai

专题命中 跨模态检索 ：multimodal(abstract)；cross-modal(abstract)

Comments ICLR 2024

Multi-Modality is All You Need for Transferable Recommender Systems

Youhua Li, Hanwen Du, Yongxin Ni, Pengpeng Zhao, Qi Guo, Fajie Yuan, Xiaofang Zhou

专题命中 跨模态检索 ：multi-modal(abstract)；cross-modal(abstract)

Comments ICDE'24 Accepted