扩散增强检索：一种交互式文本到图像检索的免训练方法
Diffusion Augmented Retrieval: A Training-Free Approach to Interactive Text-to-Image Retrieval
AI总结：
针对交互式文本到图像检索中微调多模态大语言模型成本高且泛化差的问题，提出免训练框架DAR，利用扩散模型和LLM对话细化生成中间表示，在复杂查询上表现更优。
摘要：
交互式文本到图像检索（I-TIR）是电子商务和教育等领域中众多前沿服务的重要推动力。然而，当前方法依赖于微调的多模态大语言模型（MLLMs），其训练和更新成本高昂，且泛化能力较差。后一个问题尤为令人担忧，因为：1）微调缩小了MLLMs的预训练分布，从而降低了泛化能力；2）I-TIR引入了不断增加的查询多样性和复杂性。因此，I-TIR解决方案极有可能遇到在任何训练数据集中都未能得到充分表示的查询和图像。为了解决这一问题，我们提出利用扩散模型（DMs）进行文本到图像的映射，以避免对MLLMs进行微调，同时在复杂查询上保持稳健的性能。具体而言，我们引入了扩散增强检索（DAR），这是一个通过基于LLM的对话细化和DMs生成多个中间表示的框架，从而产生对用户信息需求更丰富的描绘。这种增强的表示有助于更准确地识别语义和视觉相关的图像。在四个基准上的广泛实验表明，对于简单查询，DAR取得了与微调I-TIR模型相当的结果，且不会产生其调优开销。此外，随着查询通过额外的对话轮次变得更加复杂，DAR在十轮后的Hits@10指标上超越了微调I-TIR模型高达7.61%，这表明其对更复杂查询具有更好的泛化能力。
英文摘要：
Interactive Text-to-image retrieval (I-TIR) is an important enabler for a wide range of state-of-the-art services in domains such as e-commerce and education. However, current methods rely on finetuned Multimodal Large Language Models (MLLMs), which are costly to train and update, and exhibit poor generalizability. This latter issue is of particular concern, as: 1) finetuning narrows the pretrained distribution of MLLMs, thereby reducing generalizability; and 2) I-TIR introduces increasing query diversity and complexity. As a result, I-TIR solutions are highly likely to encounter queries and images not well represented in any training dataset. To address this, we propose leveraging Diffusion Models (DMs) for text-to-image mapping, to avoid finetuning MLLMs while preserving robust performance on complex queries. Specifically, we introduce Diffusion Augmented Retrieval (DAR), a framework that generates multiple intermediate representations via LLM-based dialogue refinements and DMs, producing a richer depiction of the user's information needs. This augmented representation facilitates more accurate identification of semantically and visually related images. Extensive experiments on four benchmarks show that for simple queries, DAR achieves results on par with finetuned I-TIR models, yet without incurring their tuning overhead. Moreover, as queries become more complex through additional conversational turns, DAR surpasses finetuned I-TIR models by up to 7.61% in Hits@10 after ten turns, illustrating its improved generalization for more intricate queries.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
