FlowCIR: 通过流匹配实现零样本组合图像检索的语义传输
FlowCIR: Semantic Transport via Flow Matching for Zero-Shot Composed Image Retrieval
AI总结：
提出FlowCIR，利用条件流匹配将参考图像与指令组合成目标对齐查询嵌入，避免文本反转的信息损失，训练效率高，并引入多负向引导策略提升对否定指令的鲁棒性。
摘要：
零样本组合图像检索（ZS-CIR）旨在通过自然语言指令编辑参考图像来检索目标图像，而不依赖于领域特定的标注三元组。现有大多数ZS-CIR方法依赖于文本反转将参考图像转换为伪文本标记，然后通过文本空间中的简单拼接与指令组合，这对于细粒度语义可能是有损且脆弱的。在这项工作中，我们提出了一种新范式，即FlowCIR，它将ZS-CIR视为参考嵌入和目标嵌入之间的条件语义传输。利用条件流匹配，我们的模型学习一个轻量级传输场，该场将指令表示映射到以参考图像为条件的目标对齐查询嵌入。由于FlowCIR在预提取的VLM嵌入上操作，并且只训练一个小型传输模块而不更新图像或文本编码器，因此与先前基于文本反转的方法相比，它提供了一种计算高效的训练协议。得到的框架训练高效，所需的训练资源大约是先前基于文本反转的方法的十分之一。我们进一步发现否定和移除是VLM组合的主要失败模式。为了解决这个问题，我们提出了一种仅推理的多负向引导策略，该策略将包含否定的相对指令引导远离其否定语义，减轻了VLM有限的否定处理能力，并提高了对否定密集型查询的鲁棒性。在标准CIR基准上的大量实验表明，与最近的ZS-CIR方法相比，FlowCIR实现了强大且具有竞争力的性能。
英文摘要：
Zero-shot composed image retrieval (ZS-CIR) aims to retrieve a target image by editing a reference image with a natural-language instruction, without relying on domain-specific annotated triplets. Most existing ZS-CIR methods rely on textual inversion to translate the reference image into pseudo-text tokens and then compose them with the instruction via simple concatenation in the text space, which can be lossy and brittle for fine-grained semantics. In this work, we propose a new paradigm, namely FlowCIR, that casts ZS-CIR as conditional semantic transport between reference and target embeddings. Leveraging \emph{conditional flow matching}, our model learns a lightweight transport field that maps the instruction representation toward a target-aligned query embedding conditioned on the reference image. Since FlowCIR operates on pre-extracted VLM embeddings and trains only a small transport module without updating the image or text encoder, it offers a computationally efficient training protocol compared with prior textual-inversion-based approaches. The resulting framework is training-efficient, requiring roughly $10\times$ fewer training resources than prior textual-inversion-based approaches. We further identify negation and removal as a major failure mode of VLM-based composition. To address this, we propose an inference-only Multi-Negative Steering strategy that steers a negation-containing relative instruction away from its negated semantics, mitigating the limited negation handling of VLMs and improving robustness on negation-heavy queries. Extensive experiments on standard CIR benchmarks demonstrate that FlowCIR achieves strong and competitive performance compared with recent ZS-CIR methods.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
