ECOR：用于物体识别的可解释CLIP
ECOR: Explainable CLIP for Object Recognition
AI总结：
本文提出基于类别与理由联合概率分布的可解释性数学定义，并据此微调CLIP，实现可解释物体识别，在零样本设置中达到最先进性能，提升可信度。
摘要：
大型视觉语言模型（VLMs），如CLIP，已显著贡献于各种计算机视觉任务，包括物体识别和物体检测。其开放词汇特性增强了其价值。然而，其黑箱性质及预测中缺乏可解释性使其在关键领域中不太可信。近期，已有一些工作试图迫使VLMs为物体识别提供合理的理由，但这往往以牺牲分类准确性为代价。在本文中，我们首先基于类别与理由的联合概率分布，提出物体识别任务中可解释性的数学定义，然后利用该定义以可解释的方式对CLIP进行微调。通过对不同数据集的评估，我们的方法在可解释分类中展现了最先进的性能。值得注意的是，它在零样本设置中表现出色，展示了其适应性。这一进展改善了可解释的物体识别，增强了跨多种应用的可信度。代码将在发表后在线提供。
英文摘要：
Large Vision Language Models (VLMs), such as CLIP, have significantly contributed to various computer vision tasks, including object recognition and object detection. Their open vocabulary feature enhances their value. However, their black-box nature and lack of explainability in predictions make them less trustworthy in critical domains. Recently, some work has been done to force VLMs to provide reasonable rationales for object recognition, but this often comes at the expense of classification accuracy. In this paper, we first propose a mathematical definition of explainability in the object recognition task based on the joint probability distribution of categories and rationales, then leverage this definition to fine-tune CLIP in an explainable manner. Through evaluations of different datasets, our method demonstrates state-of-the-art performance in explainable classification. Notably, it excels in zero-shot settings, showcasing its adaptability. This advancement improves explainable object recognition, enhancing trust across diverse applications. The code will be made available online upon publication.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
