从像素到帖子：基于检索的时尚描述与标签生成
From Pixels to Posts: Retrieval-Augmented Fashion Captioning and Hashtag Generation
AI总结：
本文提出一种基于检索的时尚描述与标签生成方法，结合多件服装检测、属性推理和大语言模型，提升描述准确性和标签生成的视觉相关性。
摘要：
本文介绍了一种结合多件服装检测、属性推理和大语言模型（LLM）提示的检索增强框架，用于自动时尚描述和标签生成。系统旨在为时尚图像生成视觉相关、描述性强且风格有趣的文本，克服了端到端描述器在属性准确性及领域泛化方面的局限。该流程结合了基于YOLO的检测器用于多件服装定位、k-means聚类用于主导颜色提取，以及基于结构化产品索引的CLIP-FAISS检索模块，用于根据服装和性别属性推断织物和性别属性。这些属性，加上检索到的风格示例，会形成一个事实证据包，用于指导LLM生成类似人类的描述和语境丰富的标签。使用微调的BLIP模型作为监督基线模型进行比较。实验结果表明，YOLO检测器能够对九种服装类别获得平均平均精度（mAP@0.5）为0.71。RAG-LLM流程生成了具有表现力的属性对齐描述，并在标签生成中实现了平均属性覆盖度为0.80，50%阈值下完全覆盖。而BLIP则提供了更高的词汇重叠和较低的泛化能力。检索增强的方法在事实基础性、减少幻觉和在各种服装领域中可扩展部署方面具有巨大潜力。这些结果展示了检索增强生成作为自动化和视觉相关时尚内容生成的有效且可解释的范式。
英文摘要：
This paper introduces the retrieval-augmented framework for automatic fashion caption and hashtag generation, combining multi-garment detection, attribute reasoning, and Large Language Model (LLM) prompting. The system aims to produce visually grounded, descriptive, and stylistically interesting text for fashion imagery, overcoming the limitations of end-to-end captioners that have problems with attribute fidelity and domain generalization. The pipeline combines a YOLO-based detector for multi-garment localization, k-means clustering for dominant color extraction, and a CLIP-FAISS retrieval module for fabric and gender attribute inference based on a structured product index. These attributes, together with retrieved style examples, create a factual evidence pack that is used to guide an LLM to generate human-like captions and contextually rich hashtags. A fine-tuned BLIP model is used as a supervised baseline model for comparison. Experimental results show that the YOLO detector is able to obtain a mean Average Precision (mAP@0.5) of 0.71 for nine categories of garments. The RAG-LLM pipeline generates expressive attribute-aligned captions and achieves mean attribute coverage of 0.80 with full coverage at the 50% threshold in hashtag generation, whereas BLIP gives higher lexical overlap and lower generalization. The retrieval-augmented approach exhibits better factual grounding, less hallucination, and great potential for scalable deployment in various clothing domains. These results demonstrate the use of retrieval-augmented generation as an effective and interpretable paradigm for automated and visually grounded fashion content generation.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
