使用视觉语言模型探索零样本目标分类的细粒度零售产品判别
Exploring Fine-grained Retail Product Discrimination with Zero-shot Object Classification Using Vision-Language Models
AI总结：
本文针对智能零售中产品更迭频繁的问题，提出了细粒度产品分类数据集MIMEX，评估了视觉语言模型的零样本分类性能，并提出结合CLIP和DINOv2嵌入的集成方法及类别适应方法以提升分类效果。
摘要：
在智能零售应用中，大量的产品及其频繁的更迭需要可靠的零样本目标分类方法。零样本假设对于避免每次引入新产品或现有产品进行品牌重塑时都需要重新训练分类器至关重要。在本文中，我们做出了三项关键贡献。首先，我们引入了 MIMEX 数据集，包含 28 个不同的产品类别。与文献中的现有数据集不同，MIMEX 专注于细粒度产品分类，并包含多样化的零售产品。其次，我们在提出的 MIMEX 数据集上对最先进的视觉语言模型（VLMs）的零样本目标分类性能进行了基准测试。我们的实验表明，这些模型的细粒度分类性能不理想，凸显了对专门方法的需求。最后，我们提出了一种新的集成方法，将来自 CLIP 和 DINOv2 的嵌入与降维技术相结合，以增强分类性能。通过结合这些组件，我们的集成方法优于 VLMs，有效地捕获了对细粒度产品判别至关重要的视觉线索。此外，我们引入了一种类别适应方法，在标记数据稀缺的场景中利用有限样本进行视觉原型设计，解决了产品种类频繁变化的零售环境中的关键需求。为了鼓励对智能零售应用的零样本目标分类进行进一步研究，我们将向研究社区发布 MIMEX 数据集和基准。感兴趣的研究人员可以联系作者了解使用条款和条件的详细信息。代码可用：https://github.com/AnilOsmanTur/Zero-shot-Retail-Product-Classification。
英文摘要：
In smart retail applications, the large number of products and their frequent turnover necessitate reliable zero-shot object classification methods. The zero-shot assumption is essential to avoid the need for re-training the classifier every time a new product is introduced into stock or an existing product undergoes rebranding. In this paper, we make three key contributions. Firstly, we introduce the MIMEX dataset, comprising 28 distinct product categories. Unlike existing datasets in the literature, MIMEX focuses on fine-grained product classification and includes a diverse range of retail products. Secondly, we benchmark the zero-shot object classification performance of state-of-the-art vision-language models (VLMs) on the proposed MIMEX dataset. Our experiments reveal that these models achieve unsatisfactory fine-grained classification performance, highlighting the need for specialized approaches. Lastly, we propose a novel ensemble approach that integrates embeddings from CLIP and DINOv2 with dimensionality reduction techniques to enhance classification performance. By combining these components, our ensemble approach outperforms VLMs, effectively capturing visual cues crucial for fine-grained product discrimination. Additionally, we introduce a class adaptation method that utilizes visual prototyping with limited samples in scenarios with scarce labeled data, addressing a critical need in retail environments where product variety frequently changes. To encourage further research into zero-shot object classification for smart retail applications, we will release both the MIMEX dataset and benchmark to the research community. Interested researchers can contact the authors for details on the terms and conditions of use. The code is available: https://github.com/AnilOsmanTur/Zero-shot-Retail-Product-Classification.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
