超越线性可分上限：对齐视觉-语言模型中的表征
Beyond the Linear Separability Ceiling: Aligning Representations in VLMs
AI总结：
提出线性可分上限（LSC）诊断框架，发现VLM存在对齐差距，并通过对比目标重塑视觉流形，使模型在抽象组合推理任务上显著超越LSC。
摘要：
推进视觉-语言模型（VLM）的一个挑战是确定其在抽象推理任务（如Bongard问题）上的失败源于有缺陷的感知还是有缺陷的自顶向下推理。为了分离这些因素，我们引入了一个诊断框架，该框架以线性可分上限（LSC）为中心，即线性分类器在VLM的原始视觉嵌入上可达到的性能。将该框架应用于最先进的VLM，我们发现了一个普遍的“对齐差距”，其中大多数模型无法在生成性能上超越其表征的线性可分性。我们发现，少数超越这一上限的模型通过两种机制实现：进一步将视觉表征细化为更线性可分的形式，或执行非线性决策逻辑。我们证明，这一瓶颈并非根本限制，而是可解决的视觉对齐问题。我们的方法用对比目标增强标准的下一个词预测，以将视觉流形重塑为更一维线性的几何结构，改进图像间比较，并使模型在抽象组合推理任务上显著超越LSC。
英文摘要：
A challenge in advancing Visual-Language Models (VLMs) is determining whether their failures on abstract reasoning tasks, such as Bongard problems, stem from flawed perception or faulty top-down reasoning. To disentangle these factors, we introduce a diagnostic framework centered on the Linear Separability Ceiling (LSC), the performance achievable by a linear classifier on a VLM's raw visual embeddings. Applying this framework to state-of-the-art VLMs, we uncover a pervasive ''alignment gap'', where most models fail to generatively outperform the linear separability of their representations. We find that the few models surpassing this ceiling do so via two mechanisms: by further refining visual representations into a more linearly separable format or by executing non-linear decision logic. We demonstrate that this bottleneck is not a fundamental limitation but a solvable visual alignment issue. Our method augments standard next-token prediction with a contrastive objective to restructure the visual manifold into a more one-dimensionally linear geometry, improving image-to-image comparison and enabling models to significantly surpass the LSC on abstract compositional reasoning tasks.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
