Deep Layout of Custom-size Furniture through Multiple-domain Learning
IHome Company（IHome公司）；IBM Research Singapore（IBM新加坡研究院）；Radio Company（Radio公司）；Huazhong University of Science and Technology（华中科技大学）；Tongji University（同济大学）
Xinhan Di, Pengqian Yu, Danfeng Yang, Hong Zhu, Changyu Sun, YinDong Liu
备注：
Submitted to CV Conference. arXiv admin note: text overlap with arXiv:2006.13527
In this paper, we propose a multiple-domain model for producing a custom-size furniture layout in the interior scene. This model is aimed to support professional interior designers to produce interior decoration solutions with custom-size furniture more quickly. The proposed model combines a deep layout module, a domain attention module, a dimensional domain transfer module, and a custom-size module in the end-end training. Compared with the prior work on scene synthesis, our proposed model enhances the ability of auto-layout of custom-size furniture in the interior room. We conduct our experiments on a real-world interior layout dataset that contains $710,700$ designs from professional designers. Our numerical results demonstrate that the proposed model yields higher-quality layouts of custom-size furniture in comparison with the state-of-art model.
In the industrial interior design process, professional designers plan the size and position of furniture in a room to achieve a satisfactory design for selling. In this paper, we explore the interior scene design task as a Markov decision process (MDP), which is solved by deep reinforcement learning. The goal is to produce an accurate position and size of the furniture simultaneously for the indoor layout task. In particular, we first formulate the furniture layout task as a MDP problem by defining the state, action, and reward function. We then design the simulated environment and train rein…
Technique Center Ihome Corporation（爱家公司技术中心）；IBM Research（IBM研究院）
In the industrial interior design process, professional designers plan the furniture layout to achieve a satisfactory 3D design for selling. In this paper, we explore the interior graphics scenes design task as a Markov decision process (MDP) in 3D simulation, which is solved by multi-agent reinforcement learning. The goal is to produce furniture layout in the 3D simulation of the indoor graphics scenes. In particular, we firstly transform the 3D interior graphic scenes into two 2D simulated scenes. We then design the simulated environment and apply two reinforcement learning agents to learn t…
We present a structured graph variational autoencoder for generating the layout of indoor 3D scenes. Given the room type (e.g., living room or library) and the room layout (e.g., room elements such as floor and walls), our architecture generates a collection of objects (e.g., furniture items such as sofa, table and chairs) that is consistent with the room type and layout. This is a challenging problem because the generated scene should satisfy multiple constrains, e.g., each object must lie inside the room and two objects cannot occupy the same volume. To address these challenges, we propose a…
In real life, the decoration of 3D indoor scenes through designing furniture layout provides a rich experience for people. In this paper, we explore the furniture layout task as a Markov decision process (MDP) in virtual reality, which is solved by hierarchical reinforcement learning (HRL). The goal is to produce a proper two-furniture layout in the virtual reality of the indoor scenes. In particular, we first design a simulation environment and introduce the HRL formulation for a two-furniture layout. We then apply a hierarchical actor-critic algorithm with curriculum learning to solve the MD…
We address the problem of detecting and erasing furniture from a wide angle photograph of a room. Inpainting large regions of an indoor scene often results in geometric inconsistencies of background elements within the inpaint mask. To address this problem, we utilize perceptual information (e.g. instance segmentation, and room layout) to produce a geometrically consistent empty version of a room. We share important details to make this system viable, such as per-plane inpainting, automatic rectification, and texture refinement. We provide detailed ablation along with qualitative examples, jus…
University of Edinburgh（爱丁堡大学）；Google Research（谷歌研究院）
Efficient authoring of vast virtual environments hinges on algorithms that are able to automatically generate content while also being controllable. We propose a method to automatically generate furniture layouts for indoor environments. Our method is simple, efficient, human-interpretable and amenable to a wide variety of constraints. We model the composition of rooms into classes of objects and learn joint (co-occurrence) statistics from a database of training layouts. We generate new layouts by performing a sequence of conditional sampling steps, exploiting the statistics learned from the d…
KAUST（阿卜杜拉国王科技大学）；Adobe Research（奥多比研究院）；University College London（伦敦大学学院）
Scalable generation of furniture layouts is essential for many applications in virtual reality, augmented reality, game development and synthetic data generation. Many existing methods tackle this problem as a sequence generation problem which imposes a specific ordering on the elements of the layout making such methods impractical for interactive editing or scene completion. Additionally, most methods focus on generating layouts unconditionally and offer minimal control over the generated layouts. We propose COFS, an architecture based on standard transformer architecture blocks from language…
Alibaba Group（阿里巴巴集团）；Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）；School of Computing Science, Simon Fraser University（西蒙菲莎大学计算科学学院）
We introduce 3D-FRONT (3D Furnished Rooms with layOuts and semaNTics), a new, large-scale, and comprehensive repository of synthetic indoor scenes highlighted by professionally designed layouts and a large number of rooms populated by high-quality textured 3D models with style compatibility. From layout semantics down to texture details of individual objects, our dataset is freely available to the academic community and beyond. Currently, 3D-FRONT contains 18,968 rooms diversely furnished by 3D objects, far surpassing all publicly available scene datasets. In addition, the 13,151 furniture obj…
GOPI: Generation-Oriented 3D Pose Inference for Furniture Insertion from Single-View RGB-D Indoor Scenes
School of Computer Science and Engineering, Sun Yat-sen University（中山大学计算机科学与工程学院）；Guangdong Key Laboratory of Big Data Analysis and Processing（广东省大数据分析与处理重点实验室）；X-Era AI Lab（X-Era AI实验室）
Technique Center Ihome Corporation（艾佳家居技术中心）；IBM Research（IBM研究院）
In this paper, we propose an adversarial model for producing furniture layout for interior scene synthesis when the interior room is rotated. The proposed model combines a conditional adversarial network, a rotation module, a mode module, and a rotation discriminator module. As compared with the prior work on scene synthesis, our proposed three modules enhance the ability of auto-layout generation and reduce the mode collapse during the rotation of the interior room. We conduct our experiments on a proposed real-world interior layout dataset that contains 14400 designs from the professional de…
IHome Company（IHome公司）；IBM Research Singapore（IBM新加坡研究院）；Radio Company（Radio公司）；Huazhong University of Science and Technology（华中科技大学）；Tongji University（同济大学）
In this paper, we propose an end-end model for producing furniture layout for interior scene synthesis from the random vector. This proposed model is aimed to support professional interior designers to produce the interior decoration solutions more quickly. The proposed model combines a conditional floor-plan module of the room, a conditional graphical floor-plan module of the room and a conditional layout module. As compared with the prior work on scene synthesis, our proposed three modules enhance the ability of auto-layout generation given the dimensional category of the room. We conduct ou…
Ctrl-Room: Controllable Text-to-3D Room Meshes Generation with Layout Constraints
Hong Kong University of Science and Technology（香港理工大学）；LightIllusion, China（中国LightIllusion公司）；Alibaba Group（阿里巴巴集团）；Simon Fraser University, Canada（加拿大Simon Fraser大学）
Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）；University of Chinese Academy of Sciences（中国科学院大学）；Stanford University（斯坦福大学）；Cardiff University（卡迪夫大学）
3D indoor scenes are widely used in computer graphics, with applications ranging from interior design to gaming to virtual and augmented reality. They also contain rich information, including room layout, as well as furniture type, geometry, and placement. High-quality 3D indoor scenes are highly demanded while it requires expertise and is time-consuming to design high-quality 3D indoor scenes manually. Existing research only addresses partial problems: some works learn to generate room layout, and other works focus on generating detailed structure and geometry of individual furniture objects.…
University of Electronic Science and Technology of China（电子科技大学）；Zhejiang University（浙江大学）；Google（谷歌）
Panorama images have a much larger field-of-view thus naturally encode enriched scene context information compared to standard perspective images, which however is not well exploited in the previous scene understanding methods. In this paper, we propose a novel method for panoramic 3D scene understanding which recovers the 3D room layout and the shape, pose, position, and semantic category for each object from a single full-view panorama image. In order to fully utilize the rich context information, we design a novel graph neural network based context model to predict the relationship among ob…
FurniScene: A Large-scale 3D Room Dataset with Intricate Furnishing Scenes
Beijing University of Posts and Telecommunications（北京邮电大学）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；Centre for Artificial Intelligence and Robotics（人工智能与机器人中心）；University of Chinese Academy of Sciences（中国科学院大学）
Max Planck Institute for Intelligent Systems（马克斯·普朗克智能系统研究所）；Max Planck ETH Center for Learning Systems（马克斯·普朗克 ETH 学习系统中心）；NVIDIA（英伟达）；University of Toronto（多伦多大学）；Vector Institute（向量研究所）；University of Tübingen（蒂宾根大学）
The ability to synthesize realistic and diverse indoor furniture layouts automatically or based on partial input, unlocks many applications, from better interactive 3D tools to data synthesis for training and simulation. In this paper, we present ATISS, a novel autoregressive transformer architecture for creating diverse and plausible synthetic indoor environments, given only the room type and its floor plan. In contrast to prior work, which poses scene synthesis as sequence generation, our model generates rooms as unordered sets of objects. We argue that this formulation is more natural, as i…
CasLayout: Cascaded 3D Layout Diffusion for Indoor Scene Synthesis with Implicit Relation Modeling
MAIS, Institute of Automation, Chinese Academy of Sciences and School of Artificial Intelligence, University of Chinese Academy of Sciences（MAIS，自动化研究所，中国科学院，人工智能学院，中国科学院大学）；Tsinghua University（清华大学）；SKLMS, Academy of Mathematics and Systems Science, Chinese Academy of Sciences（系统科学研究所，中国科学院）；Microsoft Research Asia（微软亚洲研究院）
ET-SAM: Efficient Point Prompt Prediction in SAM for Unified Scene Text Detection and Layout Analysis
School of Computer Science, National Engineering Research Center for Multimedia Software, Institute of Artificial Intelligence（计算机科学学院、多媒体软件国家工程研究中心、人工智能研究院）；Hubei Key Laboratory of Multimedia（湖北多媒体重点实验室）；Network Communication Engineering, Wuhan University, Hubei, China（网络通信工程、武汉大学、湖北、中国）
PromptDLA: A Domain-aware Prompt Document Layout Analysis Framework with Descriptive Knowledge as a Cue
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；University of the Chinese Academy of Sciences（中国科学院大学）；Fanyu AI Laboratory, Zhongke Fanyu Technology Co., Ltd.（中科创达福元人工智能实验室）；Columbia University（哥伦比亚大学）
Brown University（布朗大学）；Simon Fraser University（西蒙弗雷泽大学）
Realistic 3D indoor scene datasets have enabled significant recent progress in computer vision, scene understanding, autonomous navigation, and 3D reconstruction. But the scale, diversity, and customizability of existing datasets is limited, and it is time-consuming and expensive to scan and annotate more. Fortunately, combinatorics is on our side: there are enough individual rooms in existing 3D scene datasets, if there was but a way to recombine them into new layouts. In this paper, we propose the task of generating novel 3D floor plans from existing 3D rooms. We identify three sub-tasks of …
