视觉错位揭示了视觉模型在整体形状处理中的隐藏差异
Visual Anagrams Reveal Hidden Differences in Holistic Shape Processing Across Vision Models
AI总结：
本研究通过构型形状分数揭示视觉模型在整体形状处理中的差异，表明应整合局部纹理与全局构型信息以提升模型鲁棒性。
摘要：
人类能够基于局部纹理线索和物体部分的配置来识别物体，但当前的视觉模型主要依赖局部纹理线索，导致特征脆弱且非组合性。关于形状与纹理偏倚的研究将形状和纹理表示置于对立，测量形状相对于纹理，忽略了模型（和人类）可能同时依赖两种线索的可能性，并掩盖了两种表示类型的绝对质量。因此，我们将形状评估重新定义为绝对构型能力的问题，通过构型形状分数（CSS）来操作化。CSS（i）测量识别在物体错位对中保留局部纹理但重新排列全局部分安排以描绘不同物体类别的能力。在86个卷积、变压器和混合模型中，CSS（ii）揭示了广泛的构型敏感度，完全自监督和语言对齐的变压器——例如DINOv2、SigLIP2和EVA-CLIP——占据CSS光谱的顶端。机制性探测揭示了（iii）高CSS网络依赖于长距离交互：半径控制的注意力掩码会消除性能，显示出独特的U形整合轮廓，且表征相似性分析暴露了从局部到全局编码的中层过渡。BagNet控制仍处于随机水平（iv），排除了“边框黑客”策略。最后，（v）我们展示了构型形状分数还预测了其他形状依赖的评估。总体而言，我们提出，真正稳健、可推广和类人视觉系统的路径可能不在于强制在形状和纹理之间做出人为选择，而在于建筑和学习框架中无缝整合局部纹理和全局构型形状。
英文摘要：
Humans are able to recognize objects based on both local texture cues and the configuration of object parts, yet contemporary vision models primarily harvest local texture cues, yielding brittle, non-compositional features. Work on shape-vs-texture bias has pitted shape and texture representations in opposition, measuring shape relative to texture, ignoring the possibility that models (and humans) can simultaneously rely on both types of cues, and obscuring the absolute quality of both types of representation. We therefore recast shape evaluation as a matter of absolute configural competence, operationalized by the Configural Shape Score (CSS), which (i) measures the ability to recognize both images in Object-Anagram pairs that preserve local texture while permuting global part arrangement to depict different object categories. Across 86 convolutional, transformer, and hybrid models, CSS (ii) uncovers a broad spectrum of configural sensitivity with fully self-supervised and language-aligned transformers -- exemplified by DINOv2, SigLIP2 and EVA-CLIP -- occupying the top end of the CSS spectrum. Mechanistic probes reveal that (iii) high-CSS networks depend on long-range interactions: radius-controlled attention masks abolish performance showing a distinctive U-shaped integration profile, and representational-similarity analyses expose a mid-depth transition from local to global coding. A BagNet control remains at chance (iv), ruling out "border-hacking" strategies. Finally, (v) we show that configural shape score also predicts other shape-dependent evals. Overall, we propose that the path toward truly robust, generalizable, and human-like vision systems may not lie in forcing an artificial choice between shape and texture, but rather in architectural and learning frameworks that seamlessly integrate both local-texture and global configural shape.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
