arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2603.14610cs.CVeess.IV
收藏

使分类器具备语义不变性:分析分类器中的语义不变性

Make it SING: Analyzing Semantic Invariants in Classifiers

Viterbi Faculty of Electrical and Computer Engineering(电气与计算机工程学院)

Harel Yadid, Meir Yossef Levi, Roy Betser, Guy Gilboa

AI总结:

本文提出SING方法,通过构建等价图像并赋予语义解释,揭示分类器中语义不变性,分析不同模型在保持语义方面的差异。

摘要:

所有分类器,包括最先进的视觉模型,都具有不变性,部分源于其线性映射的几何结构。这些不变性存在于分类器的零空间中,诱导出映射到相同输出的等价输入集。现有方法难以提供人类可解释的信息。为此,本文提出语义解释零空间几何(SING)方法,通过构建等价图像并赋予语义解释,利用网络特征到多模态视觉语言模型的映射,获取自然语言描述和视觉示例。SING可应用于单张图像揭示局部不变性,或应用于图像集进行统计分析。例如,本文方法揭示ResNet50将相关语义属性泄露到零空间,而DinoViT在保持不变空间中的类语义方面表现更优。

英文摘要:

All classifiers, including state-of-the-art vision models, possess invariants, partially rooted in the geometry of their linear mappings. These invariants, which reside in the null-space of the classifier, induce equivalent sets of inputs that map to identical outputs. The semantic content of these invariants remains vague, as existing approaches struggle to provide human-interpretable information. To address this gap, we present Semantic Interpretation of the Null-space Geometry (SING), a method that constructs equivalent images, with respect to the network, and assigns semantic interpretations to the available variations. We use a mapping from network features to multi-modal vision language models. This allows us to obtain natural language descriptions and visual examples of the induced semantic shifts. SING can be applied to a single image, uncovering local invariants, or to sets of images, allowing a breadth of statistical analysis at the class and model levels. For example, our method reveals that ResNet50 leaks relevant semantic attributes to the null space, whereas DinoViT, a ViT pretrained with self-supervised DINO, is superior in maintaining class semantics across the invariant space.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑