Pic2Word：面向零样本组合图像检索的图片到词语映射
Pic2Word: Mapping Pictures to Words for Zero-shot Composed Image Retrieval
AI总结：
本文研究零样本组合图像检索任务，提出仅需弱标注或未标注数据训练的Pic2Word方法，在多种任务上展现强泛化能力并超越部分监督方法。
摘要：
在组合图像检索（CIR）中，用户将查询图像与文本相结合以描述其预期目标。现有方法依赖于使用由查询图像、文本说明和目标图像组成的标注三元组来监督学习CIR模型。标注此类三元组成本高昂，阻碍了CIR的广泛应用。在本研究中，我们提出研究一项重要任务——零样本组合图像检索（ZS-CIR），其目标是在无需标注三元组进行训练的情况下构建CIR模型。为此，我们提出了一种名为Pic2Word的新方法，该方法仅需弱标注的图像-文本对和未标注图像数据集即可进行训练。与现有的监督CIR模型不同，我们在弱标注或未标注数据集上训练的模型在多种ZS-CIR任务中展现出强大的泛化能力，例如属性编辑、对象组合和领域转换。我们的方法在常见的CIR基准CIRR和Fashion-IQ上优于多种监督CIR方法。代码将在https://github.com/google-research/composed_image_retrieval公开发布。
英文摘要：
In Composed Image Retrieval (CIR), a user combines a query image with text to describe their intended target. Existing methods rely on supervised learning of CIR models using labeled triplets consisting of the query image, text specification, and the target image. Labeling such triplets is expensive and hinders broad applicability of CIR. In this work, we propose to study an important task, Zero-Shot Composed Image Retrieval (ZS-CIR), whose goal is to build a CIR model without requiring labeled triplets for training. To this end, we propose a novel method, called Pic2Word, that requires only weakly labeled image-caption pairs and unlabeled image datasets to train. Unlike existing supervised CIR models, our model trained on weakly labeled or unlabeled datasets shows strong generalization across diverse ZS-CIR tasks, e.g., attribute editing, object composition, and domain conversion. Our approach outperforms several supervised CIR methods on the common CIR benchmark, CIRR and Fashion-IQ. Code will be made publicly available at https://github.com/google-research/composed_image_retrieval.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
