FreestyleRet：从风格多样化查询中检索图像
FreestyleRet: Retrieving Images from Style-Diversified Queries
AI总结：
针对用户以文本、草图、低分辨率和艺术等多元风格表达检索意图的需求，本文提出新任务、Diverse-Style Retrieval 数据集及基于 Gram Matrix 与风格初始化提示调优的轻量框架。
摘要：
图像检索旨在根据给定查询检索相应的图像。在应用场景中，用户倾向于通过各种查询风格来表达其检索意图。然而，当前的检索任务主要集中于文本查询检索的探索，导致检索查询选项有限，并可能在用户意图中产生歧义或偏差。本文提出了基于风格多样化查询的图像检索任务，使基于各种查询风格的检索成为可能。为支持这一新设定，我们提出了首个 Diverse-Style Retrieval 数据集，涵盖文本、草图、低分辨率和艺术等多种查询风格。我们还提出了一个轻量级的风格多样化检索框架。对于各种查询风格输入，我们应用 Gram Matrix 提取查询的纹理特征，并将其聚类到具有风格特定基的风格空间中。随后，我们采用风格初始化提示调优模块，使视觉编码器能够理解查询的纹理和风格信息。实验表明，采用风格初始化提示调优策略的模型在风格多样化检索任务上优于现有检索模型。此外，风格多样化查询（草图+文本、艺术+文本等）可在我们的模型中同时检索。来自其他查询的辅助信息增强了相应查询内的检索性能。
英文摘要：
Image Retrieval aims to retrieve corresponding images based on a given query. In application scenarios, users intend to express their retrieval intent through various query styles. However, current retrieval tasks predominantly focus on text-query retrieval exploration, leading to limited retrieval query options and potential ambiguity or bias in user intention. In this paper, we propose the Style-Diversified Query-Based Image Retrieval task, which enables retrieval based on various query styles. To facilitate the novel setting, we propose the first Diverse-Style Retrieval dataset, encompassing diverse query styles including text, sketch, low-resolution, and art. We also propose a light-weighted style-diversified retrieval framework. For various query style inputs, we apply the Gram Matrix to extract the query's textural features and cluster them into a style space with style-specific bases. Then we employ the style-init prompt tuning module to enable the visual encoder to comprehend the texture and style information of the query. Experiments demonstrate that our model, employing the style-init prompt tuning strategy, outperforms existing retrieval models on the style-diversified retrieval task. Moreover, style-diversified queries~(sketch+text, art+text, etc) can be simultaneously retrieved in our model. The auxiliary information from other queries enhances the retrieval performance within the respective query.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
