Instruction driven image editing with standard CLIP text encoders often fails in complex scenes with many objects. We present QL-Adapter, a framework for multiple object editing that tackles two challenges: enforcing object counts and spatial layouts, and accommodating diverse categories. QL-Adapter consists of two core modules: the Image-Layout Fusion Module (ILFM) and the Cross-Modal Augmentation Module (CMAM). ILFM fuses layout priors with ViT patch tokens from the CLIP image encoder to strengthen spatial structure understanding. CMAM injects image features into the text branch to enrich textual embeddings and improve instruction following. We further build QL-Dataset, a benchmark that spans broad category, layout, and count variations, and define the task of quantity and layout consistent image editing (QL-Edit). Extensive experiments show that QL-Adapter achieves state of the art performance on QL-Edit and significantly outperforms existing models.
Fudan University（复旦大学）；China University of Petroleum（中国石油大学）
Semantic image editing utilizes local semantic label maps to generate the desired content in the edited region. A recent work borrows SPADE block to achieve semantic image editing. However, it cannot produce pleasing results due to style discrepancy between the edited region and surrounding pixels. We attribute this to the fact that SPADE only uses an image-independent local semantic layout but ignores the image-specific styles included in the known pixels. To address this issue, we propose a style-preserved modulation (SPM) comprising two modulations processes: The first modulation incorporat…
Ctrl-Room: Controllable Text-to-3D Room Meshes Generation with Layout Constraints
Hong Kong University of Science and Technology（香港理工大学）；LightIllusion, China（中国LightIllusion公司）；Alibaba Group（阿里巴巴集团）；Simon Fraser University, Canada（加拿大Simon Fraser大学）
Computer Vision Lab ETH Zürich（苏黎世联邦理工学院计算机视觉实验室）；KU Leuven（鲁汶大学）
Attribute image manipulation has been a very active topic since the introduction of Generative Adversarial Networks (GANs). Exploring the disentangled attribute space within a transformation is a very challenging task due to the multiple and mutually-inclusive nature of the facial images, where different labels (eyeglasses, hats, hair, identity, etc.) can co-exist at the same time. Several works address this issue either by exploiting the modality of each domain/attribute using a conditional random vector noise, or extracting the modality from an exemplary image. However, existing methods cann…
Latent space exploration is a technique that discovers interpretable latent directions and manipulates latent codes to edit various attributes in images generated by generative adversarial networks (GANs). However, in previous work, spatial control is limited to simple transformations (e.g., translation and rotation), and it is laborious to identify appropriate latent directions and adjust their parameters. In this paper, we tackle the problem of editing the StyleGAN image layout by annotating the image directly. To do so, we propose an interactive framework for manipulating latent codes in ac…
Magic Fixup: Streamlining Photo Editing by Watching Dynamic Videos
University of Maryland \& Adobe College Park MD USA；Adobe San Jose CA USA；Adobe Seattle CA USA；Adobe San Francisco CA USA；University of Maryland \& Adobe；Adobe
Refaçade: Editing Object with Given Reference Texture
University of Electronic Science and Technology of China（电子科技大学）；The Hong Kong Polytechnic University（香港理工大学）；The Chinese University of Hong Kong（香港中文大学）；IntelliFusion Inc.（云天励飞公司）；Astribot Inc.（Astribot公司）
UTDesign: A Unified Framework for Stylized Text Editing and Generation in Graphic Design Images
Wangxuan Institute of Computer Technology, Peking University（北京大学王轩计算机技术研究所）；State Key Laboratory of General Artificial Intelligence, Peking University（北京大学通用人工智能国家重点实验室）
Despite the recent progress of generative adversarial networks (GANs) at synthesizing photo-realistic images, producing complex urban scenes remains a challenging problem. Previous works break down scene generation into two consecutive phases: unconditional semantic layout synthesis and image synthesis conditioned on layouts. In this work, we propose to condition layout generation as well for higher semantic control: given a vector of class proportions, we generate layouts with matching composition. To this end, we introduce a conditional framework with novel architecture designs and learning …
Person image synthesis, e.g., pose transfer, is a challenging problem due to large variation and occlusion. Existing methods have difficulties predicting reasonable invisible regions and fail to decouple the shape and style of clothing, which limits their applications on person image editing. In this paper, we propose PISE, a novel two-stage generative model for Person Image Synthesis and Editing, which is able to generate realistic person images with desired poses, textures, or semantic layouts. For human pose transfer, we first synthesize a human parsing map aligned with the target pose to r…
TaleCrafter: Interactive Story Visualization with Multiple Characters
Tsinghua Shenzhen International Graduate School, Tsinghua University（清华大学深圳国际研究生院，清华大学）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；Tencent AI Lab（腾讯人工智能实验室）；Hong Kong University of Science and Technology（香港科技大学）
VideoDirector: Precise Video Editing via Text-to-Video Models
Shenzhen Campus of Sun Yat-sen University（中山大学深圳校区）；Sun Yat-sen University（中山大学）；Tsinghua University（清华大学）；National University of Defense Technology（国防科技大学）
CalliMaster: Mastering Page-level Chinese Calligraphy via Layout-guided Spatial Planning
The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；Faculty of Engineering and IT, University of Technology Sydney（悉尼大学工程与信息学院）；Xiamen University（厦门大学）；The Hong Kong University of Science and Technology（香港科技大学）
Generating photorealistic images with controllable camera pose and scene contents is essential for many applications including AR/VR and simulation. Despite the fact that rapid progress has been made in 3D-aware generative models, most existing methods focus on object-centric images and are not applicable to generating urban scenes for free camera viewpoint control and scene editing. To address this challenging task, we propose UrbanGIRAFFE, which uses a coarse 3D panoptic prior, including the layout distribution of uncountable stuff and countable objects, to guide a 3D-aware generative model.…
Anchor Token Matching: Implicit Structure Locking for Training-free AR Image Editing
VCIP, College of Computer Science, Nankai University（VCIP，计算机科学学院，南开大学）；NKIARI, Shenzhen Futian（NKIARI，深圳福田）；Computer Vision Center, Universitat Autònoma de Barcelona（计算机视觉中心，巴塞罗那自治大学）；City University of Hong Kong (Dongguan)（香港城市大学（东莞））
Realistic and Controllable 3D Gaussian-Guided Object Editing for Driving Video Generation
School of Vehicle and Mobility, and State Key Laboratory of Intelligent Green Vehicle and Mobility, Tsinghua University（车辆与移动系统学院，智能绿色车辆与移动系统国家重点实验室，清华大学）；WUWEN AI；PhiGent Robotics
