双稳态图像描述中的视觉-语言不对称性
Vision-Language Asymmetry in Bistable Image Captioning
AI总结：
通过83个双稳态刺激的行为基线和稀疏自编码器分析，发现视觉塔中同时激活两种解释，但因果干预仅能翻转默认主导刺激的描述，揭示视觉表征与语言承诺之间的不对称性。
摘要：
维特根斯坦的鸭兔图对视觉-语言模型提出了一个问题：当一个模型对模糊图像进行描述时，模型中的哪一部分决定了对某一方面的承诺？我们通过83个双稳态刺激的3,320次生成行为基线来解决这个问题，该基线在中性提示与强制选择提示下揭示了三种状态（默认主导、强制主导、强制平衡），然后使用我们在LLaVA-1.6-7B实际使用的CLIP层上训练的TopK稀疏自编码器（验证EV 0.93）来探测底层表示。在69个具有每方面特征池的双稳态刺激中，72%（50/69）在视觉塔处显示两个池同时激活，包括12/12的默认主导鸭/兔和7/8的强制平衡年轻/老年。在CLIP层22进行因果干预可以在默认主导刺激上翻转描述（在流畅性保护下兔翻转率为33%），但在任何测试系数下都无法翻转强制平衡年轻/老年的描述，尽管其视觉侧存在叠加。主导瓶颈位于视觉塔下游；视觉侧表示与语言侧承诺之间的差距是“看见”与“看作”区别的经验把手。我们还指出一个方法论注意事项：基于TopK SAE输出的秩统计需要经过结校正的排序以避免无声的行序偏差。
英文摘要：
Wittgenstein's duck-rabbit poses a question for vision-language models: when a model captions an ambiguous image, where in the model is the commitment to one aspect made? We address this with a 3,320-generation behavioral baseline over 83 bistable stimuli that surfaces three regimes (default-dominant, force-dominant, force-balanced) under neutral vs forced-choice prompting, then probe the underlying representations using a TopK sparse autoencoder we train on the CLIP layer that LLaVA-1.6-7B actually consumes (validation EV 0.93). Across 69 bistable stimuli with both per-aspect feature pools available, 72% (50/69) show simultaneous activation of both pools at the vision tower, including 12/12 default-dominant duck/rabbit and 7/8 force-balanced young/old. Causal steering at CLIP layer 22 flips captions on default-dominant stimuli (33% rabbit-flip rate under a fluency guard) but cannot flip captions on force-balanced young/old at any tested coefficient, despite their vision-side superposition. The dominance bottleneck lives downstream of the vision tower; the gap between vision-side representation and language-side commitment is an empirical handle on the seeing/seeing-as distinction. We also flag a methodological note: rank-based statistics on TopK SAE outputs require tie-corrected ranking to avoid silent row-order bias.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
