视觉Transformer表现出类人偏差：方向与颜色选择性、类别知觉及相变的证据
Vision Transformers Exhibit Human-Like Biases: Evidence of Orientation and Color Selectivity, Categorical Perception, and Phase Transitions
AI总结：
本研究探究视觉Transformer（ViTs）是否具类人偏差，通过合成数据集分析LoRA微调ViTs，发现其存在倾斜效应、颜色依赖误差、类人颜色聚类及相变现象，揭示偏差源于预训练和架构约束而非仅下游数据。
摘要：
本研究探究了视觉Transformer（ViTs）是否会形成与人类大脑相似的方向和颜色偏差。我们使用在噪声水平、角度、长度、宽度和颜色上具有可控变化的合成数据集，分析了经LoRA微调的ViTs的行为。研究结果揭示了四个关键见解：首先，ViTs表现出倾斜效应，在所有条件下180度（水平）时的角度预测误差最低。其次，角度预测误差因颜色而异：偏蓝色调的误差最高，偏黄色调的误差最低。此外，角度预测误差的聚类分析显示，ViTs对颜色的分组方式与人类知觉类别一致。除方向和颜色偏差外，我们还观察到相变现象：尽管所有条件下均出现两次相变，但当颜色作为额外数据属性纳入时，训练损失曲线的相变出现延迟。最后，我们发现特定层的注意力头会自然发展出专门能力，无论下游任务如何，都能充当任务无关的特征提取器。这些观察表明，偏差和属性主要源于原始数据集的预训练（其塑造了模型的基础表示）以及视觉Transformer固有的架构约束，而非仅由下游数据统计决定。
英文摘要：
This study explored whether Vision Transformers (ViTs) developed orientation and color biases similar to those observed in the human brain. Using synthetic datasets with controlled variations in noise levels, angles, lengths, widths, and colors, we analyzed the behavior of ViTs fine-tuned with LoRA. Our findings revealed four key insights: First, ViTs exhibited an oblique effect showing the lowest angle prediction errors at 180 deg (horizontal) across all conditions. Second, angle prediction errors varied by color. Errors were highest for bluish hues and lowest for yellowish ones. Additionally, clustering analysis of angle prediction errors showed that ViTs grouped colors in a way that aligned with human perceptual categories. In addition to orientation and color biases, we observed phase transition phenomena. While two phase transitions occurred consistently across all conditions, the training loss curves exhibited delayed transitions when color was incorporated as an additional data attribute. Finally, we observed that attention heads in certain layers inherently develop specialized capabilities, functioning as task-agnostic feature extractors regardless of the downstream task. These observations suggest that biases and properties arise primarily from pre-training on the original dataset which shapes the model's foundational representations and the inherent architectural constraints of the vision transformer, rather than being solely determined by downstream data statistics.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
