基于附加注意组合学习的文本反馈图像搜索
Image Search with Text Feedback by Additive Attention Compositional Learning
摘要:
具有文本反馈的有效图像检索将影响一系列现实世界的应用,例如电子商务。给定源图像和描述对该图像的期望修改的文本反馈,目标是通过组成多模态(图像-文本)查询来检索与源相似但满足给定修改的目标图像。我们提出了一种新的解决方案,添加剂注意组合学习(AACL),它使用了多模态转换器为基础的架构,并有效地模拟图像-文本上下文。具体来说,我们提出了一种新的基于附加注意力的图像-文本合成模块,可以无缝地插入到深度神经网络中。我们还介绍了一个新的具有挑战性的基准来自Shopping 100 k数据集。AACL在三个大规模数据集(FashionIQ,Fashion 200 k和Shopping 100 k)上进行了评估,每个数据集都有很强的基线。大量的实验表明,AACL在所有三个数据集上都取得了新的最先进的结果。
英文摘要:
Effective image retrieval with text feedback stands to impact a range of real-world applications, such as e-commerce. Given a source image and text feedback that describes the desired modifications to that image, the goal is to retrieve the target images that resemble the source yet satisfy the given modifications by composing a multi-modal (image-text) query. We propose a novel solution to this problem, Additive Attention Compositional Learning (AACL), that uses a multi-modal transformer-based architecture and effectively models the image-text contexts. Specifically, we propose a novel image-text composition module based on additive attention that can be seamlessly plugged into deep neural networks. We also introduce a new challenging benchmark derived from the Shopping100k dataset. AACL is evaluated on three large-scale datasets (FashionIQ, Fashion200k, and Shopping100k), each with strong baselines. Extensive experiments show that AACL achieves new state-of-the-art results on all three datasets.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。