感知-然后-规划：以布局为策略的单目3D场景布局估计
Perceive-then-Plan: Layout-as-Policy for Monocular 3D Scene Layout Estimation
AI总结：
提出Perceive-then-Plan框架，通过视觉语言模型将单目3D布局估计转化为感知与迭代规划问题，以布局为策略（LaP）学习动作序列逐步优化场景假设，生成更物理一致且与观测对齐的3D布局。
摘要：
从单张图像构建结构化的3D场景布局需要协调视觉观察与物理和空间约束，这一挑战难以仅通过直接预测来解决。在这项工作中，我们将单目3D布局估计形式化为一个带有视觉语言模型的感知-然后-规划问题，其中感知器首先定位3D对象，然后规划器通过动作迭代优化场景假设，这些动作在保持与输入图像一致性的同时提高物理合理性。我们提出布局为策略（LaP），将规划阶段视为策略学习问题：3D布局表示为结构化状态，并通过离散动作（如平移、旋转和缩放）进行优化。从几何增强感知器的观测对齐初始化开始，LaP规划器被训练生成逐步解决几何不一致性并强制实现现实空间关系的动作序列。为了实现有效学习，我们将监督轨迹初始化与基于偏好的优化相结合，使模型能够在无需显式奖励工程的情况下学习纠正行为。这种公式将布局估计从一次性预测任务转变为迭代优化过程，从而更好地处理全局约束和复杂的对象交互。实验表明，我们的方法生成的布局在物理上更连贯，与视觉观察更一致，同时自然支持场景编辑和操作等下游任务。
英文摘要：
Building structured 3D scene layouts from a single image requires reconciling visual observations with physical and spatial constraints, a challenge that is difficult to address with direct prediction alone. In this work, we formulate monocular 3D layout estimation as a perceive-then-plan problem with vision-language models, where a Perceiver first grounds the 3D objects and then a Planner iteratively refines the scene hypothesis through actions that improve physical plausibility while preserving consistency with the input image. We propose Layout-as-Policy (LaP), which casts the planning stage as a policy learning problem: 3D layouts are represented as structured states, and refined via discrete actions such as translation, rotation, and rescaling. Starting from an observation-aligned initialization with the geometry-enhanced Perceiver, the LaP Planner is trained to produce action sequences that progressively resolve geometric inconsistencies and enforce realistic spatial relations. To enable effective learning, we combine supervised trajectory initialization with preference-based optimization, allowing the model to learn corrective behaviors without requiring explicit reward engineering. This formulation transforms layout estimation from a one-shot prediction task into an iterative refinement process, enabling better handling of global constraints and complex object interactions. Experiments demonstrate that our approach produces layouts that are more physically coherent and better aligned with visual observations, while naturally supporting downstream tasks such as scene editing and manipulation.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
