面向沉浸式室内场景装饰的条件式360度图像合成
Conditional 360-degree Image Synthesis for Immersive Indoor Scene Decoration
AI总结：
针对360度室内图像条件装饰问题，提出带360感知布局生成器与场景清空循环约束的生成框架，在Structure3D上达SOTA，泛化性好且能提供沉浸式装饰效果。
摘要：
本文研究360度图像的条件式场景装饰问题。该方法以室内场景的360度背景照片为输入，生成同一场景全景视角下的装饰后图像。为此，我们提出了一种360感知的物体布局生成器，它能在360度视角下学习潜在物体向量，为输入的360度背景图像生成多样化的家具排布方案。我们利用该物体布局作为条件，引导生成对抗网络合成输入场景的装饰图像。为进一步增强模型的生成能力，我们设计了一种简单有效的场景清空器，它能移除生成的家具并得到清空后的场景，使模型学习循环约束。我们在Structure3D数据集上训练模型，实验表明该模型能生成布局可控的多样化装饰效果。该方法在Structure3D数据集上取得了当前最优性能，且在Zillow室内场景数据集上具备良好的泛化能力。用户研究证实，我们生成结果的真实图像质量与家具布局能带来沉浸式体验。我们的实现代码将对外公开。
英文摘要：
In this paper, we address the problem of conditional scene decoration for 360-degree images. Our method takes a 360-degree background photograph of an indoor scene and generates decorated images of the same scene in the panorama view. To do this, we develop a 360-aware object layout generator that learns latent object vectors in the 360-degree view to enable a variety of furniture arrangements for an input 360-degree background image. We use this object layout to condition a generative adversarial network to synthesize images of an input scene. To further reinforce the generation capability of our model, we develop a simple yet effective scene emptier that removes the generated furniture and produces an emptied scene for our model to learn a cyclic constraint. We train the model on the Structure3D dataset and show that our model can generate diverse decorations with controllable object layout. Our method achieves state-of-the-art performance on the Structure3D dataset and generalizes well to the Zillow indoor scene dataset. Our user study confirms the immersive experiences provided by the realistic image quality and furniture layout in our generation results. Our implementation will be made available.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
