基于描述的真实分类：拓展CLIP在部件属性识别上的极限
Real Classification by Description: Extending CLIP's Limits of Part Attributes Recognition
AI总结：
本研究提出零样本“真实”的基于描述的分类任务，通过ImageNet21k针对性训练与多分辨率改进CLIP架构，增强了模型细粒度部件属性识别能力，在六个流行基准及PACO数据集上取得显著性能提升。
摘要：
在本研究中，我们定义并解决零样本“真实”的基于描述的分类任务，这是一项新颖的任务，旨在评估像CLIP这样的视觉语言模型（VLM）仅根据描述性属性对物体进行分类的能力，且排除物体类别名称。该方法突显了VLM在理解复杂物体描述方面的当前局限性，推动这些模型超越单纯的物体识别。为促进这一探索，我们引入了一项新挑战，并发布了六个流行细粒度基准的描述数据，这些数据省略了物体名称，以鼓励研究界内真正的零样本学习。此外，我们提出了一种方法，通过使用ImageNet21k的多样化物体类别进行针对性训练，并结合由大型语言模型生成的丰富属性描述，来增强CLIP的属性检测能力。进一步地，我们引入了一种改进的CLIP架构，利用多分辨率来改善细粒度部件属性的检测。通过这些努力，我们拓宽了对CLIP中部件属性识别的理解，提升了其在六个流行基准以及作为广泛使用的物体属性识别基准的PACO数据集上的细粒度分类任务性能。代码可在以下网址获取：https://github.com/ethanbar11/grounding_ge_public。
英文摘要：
In this study, we define and tackle zero shot "real" classification by description, a novel task that evaluates the ability of Vision-Language Models (VLMs) like CLIP to classify objects based solely on descriptive attributes, excluding object class names. This approach highlights the current limitations of VLMs in understanding intricate object descriptions, pushing these models beyond mere object recognition. To facilitate this exploration, we introduce a new challenge and release description data for six popular fine-grained benchmarks, which omit object names to encourage genuine zero-shot learning within the research community. Additionally, we propose a method to enhance CLIP's attribute detection capabilities through targeted training using ImageNet21k's diverse object categories, paired with rich attribute descriptions generated by large language models. Furthermore, we introduce a modified CLIP architecture that leverages multiple resolutions to improve the detection of fine-grained part attributes. Through these efforts, we broaden the understanding of part-attribute recognition in CLIP, improving its performance in fine-grained classification tasks across six popular benchmarks, as well as in the PACO dataset, a widely used benchmark for object-attribute recognition. Code is available at: https://github.com/ethanbar11/grounding_ge_public.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
