发表机构
Southeast University; Singapore Management University（东南大学; 新加坡管理大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
ScenePilot是一种检索增强的生长-修复框架，通过HRAP模块检索布局先验、RMR模块进行轻量修复，实现了高效的文本驱动3D室内场景生成，提升了物理合理性等性能。
AI 中文摘要
文本驱动的3D室内场景生成已从基于数据集的布局建模发展为结合大语言模型与视觉语言模型的开放词汇合成，但现有方法仍存在局限：一次性生成器常产生几何无效的布局，繁重的事后优化成本高且不稳定，仅依赖提示词的规划器缺乏用于功能分组和对象关系的可复用布局先验。我们提出ScenePilot，一种检索增强的生长-修复框架，将场景生成为由先验引导的增量生长结合学习到的修正。给定提示词，分层检索增强规划（HRAP）模块检索房间、组和锚点级别的布局先验以支持功能组规划；文本驱动的基础生成器随后依次插入对象组，而强化多模态修复（RMR）模块在每次插入后执行轻量局部修正，并在完成后进行最终全局修复。为训练该策略，我们构建了SceneReverse-17k，这是一个修复轨迹数据集，通过对高质量3D场景的位置、旋转和缩放进行扰动，再将逆操作作为可执行修正目标而构建。该策略从渲染视图、场景状态、检索到的先验和编辑历史中预测结构化的移动-旋转-缩放动作。结合HRAP与RMR，ScenePilot为一次性生成和繁重的全场景优化提供了高效替代方案，在保持多样性的同时提升了物理合理性、功能连贯性和可控性。
英文摘要
Text-driven 3D indoor scene generation has advanced from dataset-bound layout modeling to open-vocabulary synthesis with large language and vision-language models. Yet existing methods remain limited: one-pass generators often yield geometrically invalid layouts, heavy post-hoc optimization is costly and unstable, and prompt-only planners lack reusable layout priors for functional grouping and object relations. We propose \textbf{ScenePilot}, a retrieval-augmented \textbf{Grow-and-Repair} framework that formulates scene generation as prior-guided incremental growth with learned rectification. Given a prompt, the Hierarchical Retrieval-Augmented Planning (HRAP) module retrieves room-, group-, and anchor-level layout priors to support functional group planning. A text-driven base generator then inserts object groups sequentially, while the Reinforcement Multimodal Repair (RMR) module performs lightweight local correction after each insertion and a final global repair after completion. To train this policy, we construct \textbf{SceneReverse-17k}, a repair-trajectory dataset built by perturbing high-quality 3D scenes in position, rotation, and scale, then using inverse operations as executable rectification targets. The policy predicts structured \emph{move--rotate--scale} actions from rendered views, scene state, retrieved priors, and edit history. By combining HRAP with RMR, ScenePilot offers an efficient alternative to one-shot generation and heavy full-scene optimization, improving physical plausibility, functional coherence, and controllability while preserving diversity.
