GRE-Diff: Gaussian Room Embeddings for Structured Layout Diffusion
Guangdong Provincial Key Laboratory of Visual Media and Multidimensional Intelligence, CSSE, Shenzhen University（广东省视觉媒体与多维智能重点实验室，计算机科学与软件学院，深圳大学）；School of Computer Science, University of Guelph（圭尔夫大学计算机科学学院）
Designing functional and aesthetically coherent floor plans requires exploring a vast space of possible room arrangements, a task that quickly becomes overwhelming for human designers. In this paper, we propose GRE-Diff, a controllable and interactive diffusion-based framework that automates the creation and editing of apartment floor plans under user-specified constraints.
By combining AI-generated suggestions with real-time, human-in-the-loop editing, the system enables users to specify room types, room counts, boundary shapes, and editing operations through LLM-parsed instructions or GUI-based interaction.
It then generates a diverse set of plausible and well-structured designs for refinement.
At the core of our approach is Gaussian Room Embedding (GRE), a continuous latent representation that models each room as a spatial Gaussian distribution capturing its location and extent.
Extensive experiments on the RPLAN dataset show that GRE-Diff produces high-quality, constraint-aware, and editable polygonal layouts, offering a practical step toward bridging AI-driven automation and human creativity in spatial design.
LayoutDiffusion: Controllable Diffusion Model for Layout-to-image Generation
College of Computer Science & Technology, Zhejiang University（浙江大学计算机科学与技术学院）；Polytechnic Institute, Zhejiang University（浙江大学理工学院）；ARC Lab, Tencent PCG（腾讯平台与内容事业群ARC实验室）；Shanghai Institute for Advanced Study of Zhejiang University（浙江大学上海高等研究院）；Shanghai AI Lab（上海人工智能实验室）；Zhejiang – Singapore Innovation and AI Joint Research Lab（浙新创新与人工智能联合实验室）
Layout design is ubiquitous in many applications, e.g. architecture/urban planning, etc, which involves a lengthy iterative design process. Recently, deep learning has been leveraged to automatically generate layouts via image generation, showing a huge potential to free designers from laborious routines. While automatic generation can greatly boost productivity, designer input is undoubtedly crucial. An ideal AI-aided design tool should automate repetitive routines, and meanwhile accept human guidance and provide smart/proactive suggestions. However, the capability of involving humans into th…
East China Normal University（华东师范大学）；Yiwo Technology（易沃科技）
3D room layout estimation by a single panorama using deep neural networks has made great progress. However, previous approaches can not obtain efficient geometry awareness of room layout with the only latitude of boundaries or horizon-depth. We present that using horizon-depth along with room height can obtain omnidirectional-geometry awareness of room layout in both horizontal and vertical directions. In addition, we propose a planar-geometry aware loss function with normals and gradients of normals to supervise the planeness of walls and turning of corners. We propose an efficient network, L…
Amazon Web Services（亚马逊云科技）；USC Information Sciences Institute（南加州大学信息科学研究所）；Shanghai Jiao Tong University（上海交通大学）
Layout-to-image generation refers to the task of synthesizing photo-realistic images based on semantic layouts. In this paper, we propose LayoutDiffuse that adapts a foundational diffusion model pretrained on large-scale image or text-image datasets for layout-to-image generation. By adopting a novel neural adaptor based on layout attention and task-aware prompts, our method trains efficiently, generates images with both high perceptual quality and layout alignment, and needs less data. Experiments on three datasets show that our method significantly outperforms other 10 generative models base…
Microsoft Research Asia（微软亚洲研究院）；Tsinghua University（清华大学）
Layout generation aims to synthesize realistic graphic scenes consisting of elements with different attributes including category, size, position, and between-element relation. It is a crucial task for reducing the burden on heavy-duty graphic design works for formatted scenes, e.g., publications, documents, and user interfaces (UIs). Diverse application scenarios impose a big challenge in unifying various layout generation subtasks, including conditional and unconditional generation. In this paper, we propose a Layout Diffusion Generative Model (LDGM) to achieve such unification with a single…
National Tsing Hua University（清华大学）；National Chiao Tung University（交通大学）；NEC Labs America（美国NEC实验室）
Although significant progress has been made in room layout estimation, most methods aim to reduce the loss in the 2D pixel coordinate rather than exploiting the room structure in the 3D space. Towards reconstructing the room layout in 3D, we formulate the task of 360 layout estimation as a problem of predicting depth on the horizon line of a panorama. Specifically, we propose the Differentiable Depth Rendering procedure to make the conversion from layout to depth prediction differentiable, thus making our proposed model end-to-end trainable while leveraging the 3D geometric information, withou…
STAY Diffusion: Styled Layout Diffusion Model for Diverse Layout-to-Image Generation
Bosch Center for Artificial Intelligence（博世人工智能中心）；Institute of Industrial Manufacturing and Management IFF, University of Stuttgart（斯图加特大学工业制造与管理研究所）；Fraunhofer Institute for Manufacturing Engineering and Automation IPA（弗劳恩霍夫制造工程与自动化研究所）
针对布局到图像生成，本文提出 STAY Diffusion，通过 EA Norm、自监督语义图与 SM Attention 增强全局条件和对象关系建模，提升图像质量、多样性、准确性与可控性。
Create Anything Anywhere: Layout-Controllable Personalized Diffusion Model for Multiple Subjects
MoE Key Laboratory of Brain-inspired Intelligent Perception and Cognition, University of Science and Technology of China（脑启发智能感知与认知大学科学与技术大学实验室）；Institute of Artificial Intelligence, Hefei Comprehensive National Science Center（人工智能研究所）
CreatiLayout: Siamese Multimodal Diffusion Transformer for Creative Layout-to-Image Generation
Institute of Trustworthy Embodied AI（可信具身人工智能研究院）；Fudan University（复旦大学）；Shanghai Collaborative Innovation Center of Intelligent Visual Computing（上海智能视觉计算协同创新中心）；Bytedance Intelligent Creation（字节跳动智能创作）
We present the first self-supervised method to train panoramic room layout estimation models without any labeled data. Unlike per-pixel dense depth that provides abundant correspondence constraints, layout representation is sparse and topological, hindering the use of self-supervised reprojection consistency on images. To address this issue, we propose Differentiable Layout View Rendering, which can warp a source image to the target camera pose given the estimated layout from the target image. As each rendered pixel is differentiable with respect to the estimated layout, we can now train the l…
National Tsing Hua University（国立清华大学）；KAUST（沙特阿拉伯国王科技大学）
We present a deep learning framework, called DuLa-Net, to predict Manhattan-world 3D room layouts from a single RGB panorama. To achieve better prediction accuracy, our method leverages two projections of the panorama at once, namely the equirectangular panorama-view and the perspective ceiling-view, that each contains different clues about the room layouts. Our network architecture consists of two encoder-decoder branches for analyzing each of the two views. In addition, a novel feature fusion structure is proposed to connect the two branches, which are then jointly trained to predict the 2D …
Controllable layout generation aims at synthesizing plausible arrangement of element bounding boxes with optional constraints, such as type or position of a specific element. In this work, we try to solve a broad range of layout generation tasks in a single model that is based on discrete state-space diffusion models. Our model, named LayoutDM, naturally handles the structured layout data in the discrete representation and learns to progressively infer a noiseless layout from the initial input, where we model the layout corruption process by modality-wise discrete diffusion. For conditional ge…
Xi’an Jiaotong University（西安交通大学）；Microsoft Research Asia（微软亚洲研究院）；Shanghai Jiaotong University（上海交通大学）；Beihang University（北京航空航天大学）；Microsoft（微软）
Conditional graphic layout generation, which generates realistic layouts according to user constraints, is a challenging task that has not been well-studied yet. First, there is limited discussion about how to handle diverse user constraints flexibly and uniformly. Second, to make the layouts conform to user constraints, existing work often sacrifices generation quality significantly. In this work, we propose LayoutFormer++ to tackle the above problems. First, to flexibly handle diverse constraints, we propose a constraint serialization scheme, which represents different user constraints as se…
Institute for AI Industry Research (AIR), Tsinghua University（清华大学智能产业研究院）；Intel Labs China（英特尔中国研究院）；Peking University（北京大学）
3D scene understanding from point clouds plays a vital role for various robotic applications. Unfortunately, current state-of-the-art methods use separate neural networks for different tasks like object detection or room layout estimation. Such a scheme has two limitations: 1) Storing and running several networks for different tasks are expensive for typical robotic platforms. 2) The intrinsic structure of separate outputs are ignored and potentially violated. To this end, we propose the first transformer architecture that predicts 3D objects and layouts simultaneously, using point cloud input…
Chinese University of Hong Kong（香港中文大学）；Tsinghua University（清华大学）
Deep generative models dominate the existing literature in layout pattern generation. However, leaving the guarantee of legality to an inexplicable neural network could be problematic in several applications. In this paper, we propose \tool{DiffPattern} to generate reliable layout patterns. \tool{DiffPattern} introduces a novel diverse topology generation method via a discrete diffusion model with compute-efficiently lossless layout pattern representation. Then a white-box pattern assessment is utilized to generate legal patterns given desired design rules. Our experiments on several benchmark…
Generating visual layouts is an essential ingredient of graphic design. The ability to condition layout generation on a partial subset of component attributes is critical to real-world applications that involve user interaction. Recently, diffusion models have demonstrated high-quality generative performances in various domains. However, it is unclear how to apply diffusion models to the natural representation of layouts which consists of a mix of discrete (class) and continuous (location, size) attributes. To address the conditioning layout generation problem, we introduce DLT, a joint discre…
DiffPattern-Flex: Efficient Layout Pattern Generation via Discrete Diffusion
The Department of Computer Science and Engineering, The Chinese University of Hong Kong, Hong Kong SAR（计算机科学与工程系，香港中文大学，香港特别行政区）；Tsinghua University（清华大学）
Diffusion models are able to generate photorealistic images in arbitrary scenes. However, when applying diffusion models to image translation, there exists a trade-off between maintaining spatial structure and high-quality content. Besides, existing methods are mainly based on test-time optimization or fine-tuning model for each input image, which are extremely time-consuming for practical applications. To address these issues, we propose a new approach for flexible image translation by learning a layout-aware image condition together with a text condition. Specifically, our method co-encodes …
POCI-Diff: 3D-Layout Guided Diffusion for Controllable Synthetic Surveillance Data Generation
DISI, University of Trento（DISI，特伦托大学）；Toretei S.r.l.（Toretei公司）；Visual AI Lab, MIT-IBM Watson AI Lab（视觉人工智能实验室，MIT-IBM沃森人工智能实验室）；Fondazione Bruno Kessler（布鲁诺·凯斯勒基金会）
Adversarial Supervision Makes Layout-to-Image Diffusion Models Thrive
Bosch Center for Artificial Intelligence（博世人工智能中心）；University of Siegen（锡根大学）；Max Planck Institute for Informatics（马克斯·普朗克信息学研究所）；University of Tübingen（蒂宾根大学）
University of Surrey（萨里大学）；Adobe Research（Adobe研究院）
We present Magic Layouts; a method for parsing screenshots or hand-drawn sketches of user interface (UI) layouts. Our core contribution is to extend existing detectors to exploit a learned structural prior for UI designs, enabling robust detection of UI components; buttons, text boxes and similar. Specifically we learn a prior over mobile UI layouts, encoding common spatial co-occurrence relationships between different UI components. Conditioning region proposals using this prior leads to performance gains on UI layout parsing for both hand-drawn UIs and app screenshots, which we demonstrate w…
University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；National Tsing Hua University（国立清华大学）；National Chiao Tung University（国立交通大学）；ShanghaiTech University（上海科技大学）；University of Washington（华盛顿大学）；Apple Inc.（苹果公司）；King Abdullah University of Science and Technology（阿卜杜拉国王科技大学）
Recent approaches for predicting layouts from 360 panoramas produce excellent results. These approaches build on a common framework consisting of three steps: a pre-processing step based on edge-based alignment, prediction of layout elements, and a post-processing step by fitting a 3D layout to the layout elements. Until now, it has been difficult to compare the methods due to multiple different design decisions, such as the encoding network (e.g. SegNet or ResNet), type of elements predicted (e.g. corners, wall/floor boundaries, or semantic segmentation), or method of fitting the 3D layout. T…
PatternPaint: Practical Layout Pattern Generation Using Diffusion-Based Inpainting
Dept. of Electrical & Computer Engineering, Duke University（杜克大学电气与计算机工程系）；Intel Corporation（英特尔公司）；Dept. of Electrical & Computer Engineering, TAMU, College Station（塔拉斯大学电气与计算机工程系）
