WISE：弱监督引导的多模态大语言模型图像分类逐步解释方法
WISE: Weak-Supervision-Guided Step-by-Step Explanations for Multimodal LLMs in Image Classification
AI总结：
本文提出WISE方法，在弱监督下将概念瓶颈模型表示转化为多模态思维链，为图像分类数据集增补可解释推理链，实验表明可解释性提升37%且微调后分类准确率提高。
摘要：
多模态大语言模型（MLLMs）在视觉-文本推理中展现出潜力，其中多模态思维链（MCoT）提示显著增强了可解释性。然而，现有MCoT方法依赖富含推理依据的数据集，并且主要关注对象间推理，忽视了对于图像分类至关重要的对象内理解。为弥补这一不足，我们提出WISE，一种弱监督引导的逐步解释方法，通过将概念瓶颈模型（CBMs）中基于概念的表示在弱监督下重构为简洁、可解释的推理链，为任意图像分类数据集增补MCoT。在十个数据集上的实验表明，我们生成的MCoT不仅将可解释性提升了37%，而且在用于微调MLLMs时还带来了分类准确率的提升。我们的工作桥接了基于概念的可解释性与生成式MCoT推理，为增强MLLMs在细粒度视觉理解中的能力提供了一个可泛化的框架。
英文摘要：
Multimodal Large Language Models (MLLMs) have shown promise in visual-textual reasoning, with Multimodal Chain-of-Thought (MCoT) prompting significantly enhancing interpretability. However, existing MCoT methods rely on rationale-rich datasets and largely focus on inter-object reasoning, overlooking the intra-object understanding crucial for image classification. To address this gap, we propose WISE, a Weak-supervision-guided Step-by-step Explanation method that augments any image classification dataset with MCoTs by reformulating the concept-based representations from Concept Bottleneck Models (CBMs) into concise, interpretable reasoning chains under weak supervision. Experiments across ten datasets show that our generated MCoTs not only improve interpretability by 37% but also lead to gains in classification accuracy when used to fine-tune MLLMs. Our work bridges concept-based interpretability and generative MCoT reasoning, providing a generalizable framework for enhancing MLLMs in fine-grained visual understanding.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
