GeoWorld-VLM：从世界模型中获取几何结构用于视觉-语言模型
GeoWorld-VLM: Geometry from World Models for Vision-Language Models
AI总结：
GeoWorld-VLM通过将冻结的摄像机条件视频世界模型的几何结构转移到视觉-语言模型中，提升空间关系推理能力，实验显示在两个不同架构上均提升了约4%的性能。
摘要：
现代视觉-语言模型（VLMs）在语义识别方面表现优异，但在基本空间关系如左、在、后、之间等上仍显脆弱。这一失败的原因出现在语言推理之前：视觉路径在特征提取过程中可能压缩或丢弃关键的3D结构线索，导致语言模型接收到的图像表示不足以支持可靠的空判断。我们引入GeoWorld-VLM，一种VLM侧蒸馏框架，将冻结的摄像机条件视频世界模型的几何结构转移到VLMs中。GeoWorld-VLM仅微调图像编码器和多模态投影器，使后投影器图像特征与中间世界模型表示对齐，同时保持主骨干冻结。给定图像、提示和采样的摄像机轨迹，世界模型教师将静态视觉输入转换为合成多视角空间信号。训练结合空间答案监督、教师-学生特征对齐和对原VLM的保留锚点。由于语言模型保持冻结，GeoWorld-VLM保留原始模型的语言能力，同时将空间改进归因于增强的视觉路径。为了评估所提方法的有效性和通用性，我们将GeoWorld-VLM应用于两种不同的VLM架构，并在两个骨干上观察到一致的改进。GeoWorld-VLM在What'sUp和VSR基准上分别提升了约4%的性能，表明世界模型引导的视觉对齐在模型结构和空间推理数据集上具有泛化能力。
英文摘要：
Modern Vision-Language Models (VLMs) achieve strong semantic recognition, yet remain brittle on elementary spatial relations such as left of, on, behind, and between. One cause of this failure arises before language reasoning begins: the visual pathway may compress or discard critical 3D structural cues during feature extraction, so the language model receives image representations that are already insufficient for reliable spatial judgment. We introduce GeoWorld-VLM, a VLM-side distillation framework that transfers geometric structure from frozen camera-conditioned video world models into VLMs. GeoWorld-VLM fine-tunes only the image encoder and multimodal projector, aligning post-projector image features with intermediate world-model representations while leaving the main backbone frozen. Given images, a prompt, and a sampled camera trajectory, the world-model teacher converts static visual input into a synthetic multi-view spatial signal. Training combines spatial answer supervision, teacher-student feature alignment, and a preservation anchor to the original VLM. Since the language model remains frozen, GeoWorld-VLM preserves the original model's linguistic capabilities while attributing spatial improvements to the enhanced visual pathway. To evaluate the effectiveness and generality of the proposed method, we apply GeoWorld-VLM to two distinct VLM architectures and observe consistent improvements across both backbones. GeoWorld-VLM improves performance by approximately 4 percent on both the What'sUp and VSR benchmarks, suggesting that world-model-guided visual alignment generalizes across model structures and spatial reasoning datasets.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
