UNION: 一种轻量级的目标表示用于高效零样本图像引导检索与可选文本查询
UNION: A Lightweight Target Representation for Efficient Zero-Shot Image-Guided Retrieval with Optional Textual Queries
AI总结：
UNION通过融合图像嵌入与空文本提示，实现高效零样本图像引导检索，仅需5,000训练样本即在多个基准测试中超越传统基线。
摘要：
图像引导检索与可选文本（IGROT）是一种通用检索设置，其中查询由一个锚点图像（可能带有伴随文本）组成，旨在检索语义相关的目标图像。这种 formulation 统一了两个主要任务：组合图像检索（CIR）和基于草图的图像检索（SBIR）。在本工作中，我们通过引入UNION，一种轻量级且可推广的目标表示，来解决在低数据监督下IGROT的问题。UNION融合了图像嵌入与一个空文本提示。不同于传统方法依赖固定目标特征，UNION在多模态查询中增强语义对齐，同时不需要对预训练视觉-语言模型进行架构修改。仅使用5,000个训练样本（从LlavaSCo进行CIR和Training-Sketchy进行SBIR），我们的方法在多个基准测试中取得了竞争性的结果，包括CIRCO mAP@50为38.5和Sketchy mAP@200为82.7，超过了许多高度监督的基线。这表明UNION在不同查询类型中连接视觉和语言的鲁棒性和效率。
英文摘要：
Image-Guided Retrieval with Optional Text (IGROT) is a general retrieval setting where a query consists of an anchor image, with or without accompanying text, aiming to retrieve semantically relevant target images. This formulation unifies two major tasks: Composed Image Retrieval (CIR) and Sketch-Based Image Retrieval (SBIR). In this work, we address IGROT under low-data supervision by introducing UNION, a lightweight and generalisable target representation that fuses the image embedding with a null-text prompt. Unlike traditional approaches that rely on fixed target features, UNION enhances semantic alignment with multimodal queries while requiring no architectural modifications to pretrained vision-language models. With only 5,000 training samples - from LlavaSCo for CIR and Training-Sketchy for SBIR - our method achieves competitive results across benchmarks, including CIRCO mAP@50 of 38.5 and Sketchy mAP@200 of 82.7, surpassing many heavily supervised baselines. This demonstrates the robustness and efficiency of UNION in bridging vision and language across diverse query types.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
