HouseCrafter：利用2D扩散模型将平面图提升为3D场景
HouseCrafter: Lifting Floorplans to 3D Scenes with 2D Diffusion Model
AI总结：
HouseCrafter通过适配2D扩散模型自回归生成一致的RGB-D图像，将平面图提升为房屋尺度的高质量3D室内场景，并在3D-Front上验证了有效性。
摘要：
我们提出了HouseCrafter，这是一种新颖的方法，能够将平面图提升为完整的大型3D室内场景（例如，一栋房屋）。我们的关键思路是适配一个在大规模网络图像上训练的2D扩散模型，以在场景的不同位置生成一致的多视角彩色（RGB）和深度（D）图像。具体来说，RGB-D图像基于平面图沿采样位置以批量方式自回归生成，其中先前生成的图像被用作扩散模型的条件，以在附近位置生成图像。扩散模型中的全局平面图和注意力设计确保了生成图像的一致性，从而可以重建出3D场景。通过在3D-Front数据集上的广泛评估，我们证明HouseCrafter能够生成高质量的房屋尺度3D场景。消融研究也验证了不同设计选择的有效性。我们将发布代码和模型权重。项目页面：https://neu-vi.github.io/houseCrafter/
英文摘要：
We introduce HouseCrafter, a novel approach that can lift a floorplan into a complete large 3D indoor scene (e.g., a house). Our key insight is to adapt a 2D diffusion model, which is trained on web-scale images, to generate consistent multi-view color (RGB) and depth (D) images across different locations of the scene. Specifically, the RGB-D images are generated autoregressively in a batch-wise manner along sampled locations based on the floorplan, where previously generated images are used as condition to the diffusion model to produce images at nearby locations. The global floorplan and attention design in the diffusion model ensures the consistency of the generated images, from which a 3D scene can be reconstructed. Through extensive evaluation on the 3D-Front dataset, we demonstrate that HouseCraft can generate high-quality house-scale 3D scenes. Ablation studies also validate the effectiveness of different design choices. We will release our code and model weights. Project page: https://neu-vi.github.io/houseCrafter/
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
