分解视觉分类：评估VLMs中基于树的推理
Decomposing Visual Classification: Assessing Tree-Based Reasoning in VLMs
AI总结：
本文研究结构化基于树的推理能否提升VLMs在细粒度视觉分类中的性能，发现尽管模型能理解树知识，但该方法仍逊色于零样本提示，而添加LLM生成的描述可提升两者性能，揭示了结构化推理的局限性。
摘要：
视觉语言模型（VLMs）在零样本视觉分类中表现出色，但其在细粒度任务和大型层次标签空间上的性能仍未被充分研究。本文研究了结构化的、基于树的推理是否能提升VLM性能。我们引入了一个框架，使用决策树将分类分解为可解释的决策，并在细粒度（GTSRB）和粗粒度（CIFAR-10）数据集上对其进行评估。尽管模型在理解树知识方面达到了98.2%的准确率，但基于树的推理始终逊色于标准的零样本提示。我们还探索了使用LLM生成的类别和图像描述来增强树提示，以改善对齐。添加的描述提升了基于树方法和零样本方法的性能。我们的发现突出了视觉分类中结构化推理的局限性，并为设计更具可解释性的VLM系统提供了见解。
英文摘要：
Vision language models (VLMs) excel at zero-shot visual classification, but their performance on fine-grained tasks and large hierarchical label spaces is understudied. This paper investigates whether structured, tree-based reasoning can enhance VLM performance. We introduce a framework that decomposes classification into interpretable decisions using decision trees and evaluates it on fine-grained (GTSRB) and coarse-grained (CIFAR-10) datasets. Although the model achieves 98.2% accuracy in understanding the tree knowledge, tree-based reasoning consistently underperforms standard zero-shot prompting. We also explore enhancing the tree prompts with LLM-generated classes and image descriptions to improve alignment. The added description enhances the performance of the tree-based and zero-shot methods. Our findings highlight limitations of structured reasoning in visual classification and offer insights for designing more interpretable VLM systems.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
