动态概念:可解释视频分类的时序概念瓶颈模型
Concepts in Motion: Temporal Concept Bottleneck Model for Interpretable Video Classification
AI总结:
本文提出MoTIF模型,通过时间接地的概念激活序列进行视频分类,利用时序自注意力机制建模概念的出现时间和贡献,提升可解释性。
摘要:
本文提出MoTIF模型,通过时间接地的概念激活序列进行视频分类,利用时序自注意力机制建模概念的出现时间和贡献,提升可解释性。
英文摘要:
Concept Bottleneck Models (CBMs) enable interpretable image classification by structuring predictions around human-understandable concepts, but extending this paradigm to video remains challenging due to the difficulty of extracting concepts and modeling them over time. In this paper, we introduce MoTIF (Moving Temporal Interpretable Framework), a transformer-based concept architecture that operates on sequences of temporally grounded concept activations, by employing per-concept temporal self-attention to model when individual concepts recur and how their temporal patterns contribute to predictions. Central to the framework is a class-conditioned VLM-based concept discovery module that extracts object- and action-centric textual concepts from training videos, yielding temporally expressive concept sets without manual concept annotation. Across multiple video benchmarks, this combination improves over global concept bottlenecks and remains competitive within the interpretable concept-bottleneck setting, while narrowing the gap to strong black-box video baselines that we report as contextual references. Code available at github.com/patrick-knab/MoTIF.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。