arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2607.04665cs.CV

DiCE-CIR:用于高效零样本合成图像检索的直接合成学习

DiCE-CIR: Direct Composition Learning for Efficient Zero-Shot Composed Image Retrieval

Institute of Information & Communications Technology Planning & Evaluation (IITP)(信息通信技术规划与评估研究所(IITP));Department of Artificial Intelligence, Korea University(韩国大学人工智能系)

Gwang-Ho Na, Ho-Joong Kim, Seong-Whan Lee

AI总结:

研究零样本合成图像检索问题,提出直接合成学习方法DiCE-CIR,直接组合参考图像和编辑文本预测合成查询表示,用大语言模型自动构建训练样本,训练轻量级模块,实验表明其性能优且效率高。

摘要:

零样本合成图像检索(ZS-CIR)旨在从由参考图像和描述所需修改的编辑文本组成的多模态查询中检索目标图像。最近的ZS-CIR研究依赖于基于投影的方法,将参考图像映射到文本嵌入空间中的伪单词令牌。然而,这种方法需要额外的投影和重新编码步骤,增加了训练复杂性,降低了效率,并在训练和推理之间引入了差异。在本文中,我们提出了DiCE-CIR,一种直接合成学习方法,通过直接组合参考图像和编辑文本预测合成查询表示。为了在没有人工注释三元组的情况下实现可扩展训练,我们使用大语言模型从大规模图像-标题对中自动构建合成训练样本。基于这些样本,我们训练一个轻量级合成模块,其目标是促进与目标的对齐、编辑一致的语义转换和检索可辨别性。我们在ZS-CIR基准上进行了广泛的实验,表明DiCE-CIR在CIRCO上实现了领先的性能,在CIRR上实现了有竞争力的性能,同时保持了高计算效率。

英文摘要:

Zero-shot composed image retrieval (ZS-CIR) aims to retrieve a target image from a multimodal query consisting of a reference image and an edit text describing the desired modification. Recent ZS-CIR studies have relied on projection-based methods that map a reference image into pseudo-word tokens in the text embedding space. However, such methods require additional projection and re-encoding steps, increasing training complexity, reducing efficiency, and introducing a discrepancy between training and inference. In this paper, we propose DiCE-CIR, a direct composition learning method that predicts composed query representations by directly composing a reference image and an edit text. To enable scalable training without manually annotated triplets, we automatically construct compositional training samples from large-scale image-caption pairs using a large language model. Based on these samples, we train a lightweight composition module with objectives that promote alignment with the target, edit-consistent semantic transformation, and retrieval discriminability. We conduct extensive experiments on ZS-CIR benchmarks and show that DiCE-CIR achieves state-of-the-art performance on CIRCO and competitive performance on CIRR while maintaining high computational efficiency.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑