基于对象中心3D展开的视频空间推理
Video Spatial Reasoning with Object-Centric 3D Rollout
AI总结：
针对多模态大语言模型在视频空间推理中存在的查询锁定问题，提出对象中心3D展开（OCR）策略，通过结构化扰动对象3D几何结构并结合展开训练流水线，使3B参数模型在VSI-Bench上准确率达47.5%，优于7B基线模型。
摘要：
近年来，多模态大语言模型（MLLMs）在视觉-语言理解方面取得了显著进展，但实现稳健的视频空间推理——即理解动态3D场景中对象的位置、方向和对象间关系的能力——仍是一个关键的未解决挑战。现有方法主要依赖于空间接地的监督微调或强化学习，然而我们观察到这类模型常表现出查询锁定推理，仅狭隘关注提示中明确提及的对象，却忽略关键上下文线索。为解决这一局限，我们提出对象中心3D展开（OCR）策略，在训练期间对选定对象的3D几何结构引入结构化扰动。通过弱化对象特定视觉线索并将改变后的几何结构投影到2D空间，OCR迫使模型对整个场景进行整体推理。我们进一步设计了基于展开的训练流水线，联合利用原始视频和区域噪声视频优化空间推理轨迹。实验表明，我们的3B参数模型在VSI-Bench上达到47.5%的准确率，优于多个7B基线模型；消融实验证实OCR优于T-GRPO、NoisyRollout等现有展开策略。
英文摘要：
Recent advances in Multi-modal Large Language Models (MLLMs) have showcased remarkable capabilities in vision-language understanding. However, enabling robust video spatial reasoning-the ability to comprehend object locations, orientations, and inter-object relationships in dynamic 3D scenes-remains a key unsolved challenge. Existing approaches primarily rely on spatially grounded supervised fine-tuning or reinforcement learning, yet we observe that such models often exhibit query-locked reasoning, focusing narrowly on objects explicitly mentioned in the prompt while ignoring critical contextual cues. To address this limitation, we propose Object-Centric 3D Rollout (OCR), a novel strategy that introduces structured perturbations to the 3D geometry of selected objects during training. By degrading object-specific visual cues and projecting the altered geometry into 2D space, OCR compels the model to reason holistically across the entire scene. We further design a rollout-based training pipeline that jointly leverages vanilla and region-noisy videos to optimize spatial reasoning trajectories. Experiments demonstrate state-of-the-art performance: our 3B-parameter model achieves 47.5% accuracy on VSI-Bench, outperforming several 7B baselines. Ablations confirm OCR's superiority over prior rollout strategies (e.g., T-GRPO, NoisyRollout).
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
