RoomPilot: 通过多模态语义解析实现可控的室内场景合成
RoomPilot: Controllable Indoor Scene Synthesis via Multimodal Semantic Parsing
AI总结：
该研究提出RoomPilot框架，通过多模态语义解析实现可控的室内场景合成，解决了现有方法输入模态有限和生成过程隐式的问题，提高了场景结构和语义的可控性。
摘要：
生成可控的室内场景对于游戏开发、建筑可视化和具身AI应用至关重要。然而，现有方法要么只支持有限的输入模态，要么依赖隐式生成过程，限制了对场景结构和语义的精确控制。为了解决这些限制，我们引入RoomPilot，一个统一的框架，从多模态输入（包括文本描述和CAD平面图）中生成可控的室内场景。RoomPilot将异构输入映射到一个室内领域特定语言（IDSL），作为描述室内场景的结构化和可解释的语义表示。基于IDSL，RoomPilot提出一个分层合成流程，逐步在建筑、房间和物体层面组织场景，促进多房间布局中的结构一致性和功能一致性。此外，RoomPilot构建了一个经过精心挑选的资产数据集，具有丰富的语义注释，以支持高质量的场景合成，提高视觉真实感和外观一致性。广泛的实验表明，该方法在多模态理解、场景生成的细粒度可控性以及物理一致性和视觉保真度方面均有所提升，标志着可控3D室内场景合成的重要一步。代码和模型将公开。
英文摘要：
Generating controllable indoor scenes is fundamental to applications in game development, architectural visualization, and embodied AI. However, existing approaches either support a limited input modalities or rely on implicit generation processes that hinder precise control over scene structure and semantics. To address these limitations, we introduce RoomPilot, a unified framework for controllable indoor scene synthesis from multi-modal inputs, including textual descriptions and CAD floor plans. RoomPilot maps heterogeneous inputs into an Indoor Domain-Specific Language (IDSL), which serves as a structured and interpretable semantic representation for describing indoor scenes. Built upon IDSL, RoomPilot presents a hierarchical synthesis pipeline that progressively organizes scenes at the building, room, and object levels, promoting structural coherence and functional consistency across multi-room layouts. Moreover, RoomPilot constructs a curated asset dataset with rich semantic annotations to support high-quality scene synthesis, improving visual realism and appearance consistency. Extensive experiments demonstrate effective multi-modal understanding, fine-grained controllability in scene generation, and improved physical consistency and visual fidelity, marking a significant step toward controllable 3D indoor scene synthesis. Code and model will be available.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
