探索多模态大语言模型与两阶段微调在时尚图像检索中的应用
Exploring Multi-Modal Large Language Models and Two-Stage Fine-Tuning for Fashion Image Retrieval
AI总结：
提出融合多模态大语言模型（LLaVA）生成属性感知三元组，并采用两阶段微调策略增强对比学习，以解决时尚图像检索中标注数据稀缺和负采样简单的问题。
摘要：
组合图像检索通过参考图像和修改文本描述的复合查询来检索目标图像。在时尚领域，该任务需要理解颜色、图案和纹理等细微属性变化。然而，现有方法因标注数据稀缺和负采样简单而面临局限性。我们提出了一种新颖框架，该框架集成多模态大语言模型（LLaVA）以生成属性感知三元组，并引入两阶段微调策略来增强对比学习。我们利用预训练的视觉-语言模型（如CLIP-ViT/B32）生成句子级提示并与相对描述拼接，以及使用静态表示来增加负样本数量。实验结果表明，该框架增强了组合推理能力并改进了细粒度检索行为，突显了所提框架在时尚检索中的可行性和潜力。
英文摘要：
Composed image retrieval retrieves a target image using a composed query of a reference image and a modified text description. In the fashion domain, this task requires understanding subtle attribute variations such as color, pattern, and texture. However, existing approaches face limitations due to scarce annotated data and simplistic negative sampling. We propose a novel framework that integrates a multi-modal large language model (LLaVA) to generate attribute-aware triplets and introduces a two-stage fine-tuning strategy to enhance contrastive learning. We leverage pretrained vision-language models, such as CLIP-ViT/B32, to generate and concatenate sentence-level prompts with the relative caption and to scale the number of negatives using static representations. Experimental results demonstrate enhanced compositional reasoning and improved fine-grained retrieval behavior, underscoring the feasibility and potential of the proposed framework for fashion retrieval.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
