Let's Go Shopping (LGS)——面向视觉概念理解的网络规模图文数据集
Let's Go Shopping (LGS) -- Web-Scale Image-Text Dataset for Visual Concept Understanding
AI总结：
针对现有视觉数据集收集难、数据源有缺陷的问题，本文推出含1500万对图文对的电商领域LGS数据集，验证了其在分类泛化、视觉-语言双模态任务中的应用价值。
摘要：
神经网络的视觉与视觉-语言应用（如图像分类和图像描述）依赖大规模标注数据集，而这类数据集的收集过程难度不小。这项耗时的工作阻碍了大规模数据集的涌现，将研究人员和从业者的可选范围限制在少数数据集内。因此，我们寻求更高效的图像收集与标注方法。此前的研究方案从HTML替代文本中收集描述、抓取社交媒体帖子，但这些数据源存在噪声大、稀疏性强或主观性强的问题。为此，我们转向数据满足清洁性、信息性和流畅性三项标准的商业购物网站。我们推出Let's Go Shopping (LGS)数据集，这是一个包含1500万对图文对的大规模公开数据集，数据均来自公开可访问的电商网站。与现有通用域数据集相比，LGS的图像以前景物体为焦点，背景复杂度更低。我们在LGS上的实验表明，在现有基准数据集上训练的分类器无法很好地泛化到电商数据，而特定的自监督视觉特征提取器的泛化能力更强。此外，LGS高质量的电商聚焦图像及其双模态属性使其在视觉-语言双模态任务中具备优势：LGS能让图像描述模型生成更丰富的描述，还能帮助文本生成图像模型实现电商风格迁移。
英文摘要：
Vision and vision-language applications of neural networks, such as image classification and captioning, rely on large-scale annotated datasets that require non-trivial data-collecting processes. This time-consuming endeavor hinders the emergence of large-scale datasets, limiting researchers and practitioners to a small number of choices. Therefore, we seek more efficient ways to collect and annotate images. Previous initiatives have gathered captions from HTML alt-texts and crawled social media postings, but these data sources suffer from noise, sparsity, or subjectivity. For this reason, we turn to commercial shopping websites whose data meet three criteria: cleanliness, informativeness, and fluency. We introduce the Let's Go Shopping (LGS) dataset, a large-scale public dataset with 15 million image-caption pairs from publicly available e-commerce websites. When compared with existing general-domain datasets, the LGS images focus on the foreground object and have less complex backgrounds. Our experiments on LGS show that the classifiers trained on existing benchmark datasets do not readily generalize to e-commerce data, while specific self-supervised visual feature extractors can better generalize. Furthermore, LGS's high-quality e-commerce-focused images and bimodal nature make it advantageous for vision-language bi-modal tasks: LGS enables image-captioning models to generate richer captions and helps text-to-image generation models achieve e-commerce style transfer.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
