Traditional image-to-3D models often struggle with scenes containing multiple objects due to biases and occlusion complexities. To address this challenge, we present REPARO, a novel approach for compositional 3D asset generation from single images. REPARO employs a two-step process: first, it extracts individual objects from the scene and reconstructs their 3D meshes using off-the-shelf image-to-3D models; then, it optimizes the layout of these meshes through differentiable rendering techniques, ensuring coherent scene composition. By integrating optimal transport-based long-range appearance loss term and high-level semantic loss term in the differentiable rendering, REPARO can effectively recover the layout of 3D assets. The proposed method can significantly enhance object independence, detail accuracy, and overall scene coherence. Extensive evaluation of multi-object scenes demonstrates that our REPARO offers a comprehensive approach to address the complexities of multi-object 3D scene generation from single images.
Recognizing human actions is fundamentally a spatio-temporal reasoning problem, and should be, at least to some extent, invariant to the appearance of the human and the objects involved. Motivated by this hypothesis, in this work, we take an object-centric approach to action recognition. Multiple works have studied this setting before, yet it remains unclear (i) how well a carefully crafted, spatio-temporal layout-based method can recognize human actions, and (ii) how, and when, to fuse the information from layout and appearance-based models. The main focus of this paper is compositional/few-s…
Training-Free Text-to-Image Compositional Food Generation via Prompt Grafting
Elmore Family School of Electrical and Computer Engineering, Purdue University（电气与计算机工程学院，普渡大学）；Department of Systems Design Engineering, University of Waterloo（系统设计工程系，滑铁卢大学）
Layout-your-3D: Controllable and Precise 3D Generation with 2D Blueprint
Huazhong University of Science and Technology（华中科技大学）；NVIDIA（英伟达）；Wuhan University（武汉大学）；Insta360 Research（影石创新研究院）；UC Merced（加州大学默塞德分校）；Yonsei University（延世大学）
Generating photorealistic images with controllable camera pose and scene contents is essential for many applications including AR/VR and simulation. Despite the fact that rapid progress has been made in 3D-aware generative models, most existing methods focus on object-centric images and are not applicable to generating urban scenes for free camera viewpoint control and scene editing. To address this challenging task, we propose UrbanGIRAFFE, which uses a coarse 3D panoptic prior, including the layout distribution of uncountable stuff and countable objects, to guide a 3D-aware generative model.…
DM-CFO: A Diffusion Model for Compositional 3D Tooth Generation with Collision-Free Optimization
School of Computer Science and Technology, Zhejiang Gongshang University（浙江工商大学计算机科学与技术学院）；AGH University of Krakow（克拉科夫AGH大学）；Polish Ministry of Science and Higher Education（波兰教育部）；Tongxiang Institute of General Artificial Intelligence（同祥通用人工智能研究院）；State Key Laboratory of Advanced Medical Materials and Devices（先进医学材料与器件国家重点实验室）；School of Artificial Intelligence and Computer Science, Nantong University（南通大学人工智能与计算机科学学院）；Department of Computer Science, College of Computer Engineering and Sciences, Prince Sattam Bin Abdulaziz University（普莱斯·本·阿卜杜勒阿齐兹大学计算机科学系）；Department of Computer Science, Qena University（基纳大学计算机科学系）；Department of Computing Sciences, Tampere University（塔尔库大学计算科学系）；Department of Computer Science, Universidade Federal Fluminense（里约热内卢联邦大学计算机科学系）；Shining3D Tech Co., Ltd.（Shining3D科技有限公司）
Stanford University（斯坦福大学）；Facebook AI Research（Facebook人工智能研究院）；Facebook, Inc.（Facebook公司）
We introduce the GANformer2 model, an iterative object-oriented transformer, explored for the task of generative modeling. The network incorporates strong and explicit structural priors, to reflect the compositional nature of visual scenes, and synthesizes images through a sequential process. It operates in two stages: a fast and lightweight planning phase, where we draft a high-level scene layout, followed by an attention-based execution phase, where the layout is being refined, evolving into a rich and detailed picture. Our model moves away from conventional black-box GAN architectures that …
University of British Columbia（不列颠哥伦比亚大学）；The Chinese University of Hong Kong（香港中文大学）
Despite significant progress, controlled generation of complex images with interacting people remains difficult. Existing layout generation methods fall short of synthesizing realistic person instances; while pose-guided generation approaches focus on a single person and assume simple or known backgrounds. To tackle these limitations, we propose a new problem, \textbf{Persons in Context Synthesis}, which aims to synthesize diverse person instance(s) in consistent contexts, with user control over both. The context is specified by the bounding box object layout which lacks shape information, whi…
Evaluating Design Video Generation: Metrics for Compositional Fidelity
Lica World（Lica世界）；San Francisco, United States of America（美国旧金山）；ICML’26 Workshop on Human-AI Co-Creativity, Seoul, South Korea（ICML’26 人类-人工智能协同创作研讨会，韩国首尔）
University of California, San Diego（加利福尼亚大学圣地亚哥分校）；Amazon AWS（亚马逊云科技）；University of Maryland, College Park（马里兰大学学院公园分校）
We address the problem of scene layout generation for diverse domains such as images, mobile applications, documents, and 3D objects. Most complex scenes, natural or human-designed, can be expressed as a meaningful arrangement of simpler compositional graphical primitives. Generating a new layout or extending an existing layout requires understanding the relationships between these primitives. To do this, we propose LayoutTransformer, a novel framework that leverages self-attention to learn contextual relationships between layout elements and generate novel layouts in a given domain. Our frame…
TTOM: Test-Time Optimization and Memorization for Compositional Video Generation
National University of Singapore（国立新加坡大学）；University of Science and Technology of China（中国科学技术大学）；Harbin Institute of Technology (Shenzhen)（哈尔滨工业大学（深圳））
University of Virginia（弗吉尼亚大学）；IBM Thomas J. Watson Research Center（IBM托马斯·J·沃森研究中心）
In this paper, we propose Text2Scene, a model that generates various forms of compositional scene representations from natural language descriptions. Unlike recent works, our method does NOT use Generative Adversarial Networks (GANs). Text2Scene instead learns to sequentially generate objects and their attributes (location, size, appearance, etc) at every time step by attending to different parts of the input text and the current status of the generated scene. We show that under minor modifications, the proposed framework can handle the generation of different forms of scene representations, i…
School of Electrical and Computer Engineering, Peking University（北京大学电子工程学院）；Hupan Lab（鸿篇实验室）；DAMO Academy, Alibaba Group（阿里云达摩院）；Shanghai Jiao Tong University（上海交通大学）
本文提出 CoT-Diff 框架，将 MLLM 驱动的 3D 布局规划与扩散去噪过程逐步耦合，实现布局感知的文本到图像生成，在复杂场景空间准确率上超越现有最优方法 34.7%。
Xi’an Jiaotong University（西安交通大学）；Zhejiang University of Technology（浙江工业大学）；SGIT AI Lab（SGIT人工智能实验室）；China Telecom Artificial Intelligence Technology Co.Ltd（中国电信人工智能科技有限公司）；Baidu Inc（百度公司）
SPOON: Towards Coherent Compositional 3D Scene Generation from Uncalibrated Multi-view Images
The University of Hong Kong（香港大学）；Shenzhen Loop Area Institute（深圳河套学院）；The Hong Kong University of Science and Technology（香港科技大学）；Sun Yat-sen University（中山大学）；TranscEngram
Scene Graph Disentanglement and Composition for Generalizable Complex Image Generation
Shanghai Jiao Tong University（上海交通大学）；Ningbo Institute of Digital Twin, Eastern Institute of Technology（东方理工大学宁波数字孪生研究院）；University of Science and Technology of China（中国科学技术大学）；The University of Hong Kong（香港大学）
University of California, Santa Barbara（加州大学圣巴巴拉分校）；University of California, Santa Cruz（加州大学圣克鲁兹分校）；Google（谷歌）
Large-scale diffusion models have achieved state-of-the-art results on text-to-image synthesis (T2I) tasks. Despite their ability to generate high-quality yet creative images, we observe that attribution-binding and compositional capabilities are still considered major challenging issues, especially when involving multiple objects. In this work, we improve the compositional skills of T2I models, specifically more accurate attribute binding and better image compositions. To do this, we incorporate linguistic structures with the diffusion guidance process based on the controllable properties of …
