Enhancing Object Coherence in Layout-to-Image Synthesis
School of Computer Science, Fudan University（复旦大学计算机科学学院）；College of Computer Science and Technology, Zhejiang University of Technology（浙江工业大学计算机科学与技术学院）
Layout-to-image synthesis is an emerging technique in conditional image generation. It aims to generate complex scenes, where users require fine control over the layout of the objects in a scene. However, it remains challenging to control the object coherence, including semantic coherence (e.g., the cat looks at the flowers or not) and physical coherence (e.g., the hand and the racket should not be misaligned). In this paper, we propose a novel diffusion model with effective global semantic fusion (GSF) and self-similarity feature enhancement modules to guide the object coherence for this task. For semantic coherence, we argue that the image caption contains rich information for defining the semantic relationship within the objects in the images. Instead of simply employing cross-attention between captions and latent images, which addresses the highly relevant layout restriction and semantic coherence requirement separately and thus leads to unsatisfying results shown in our experiments, we develop GSF to fuse the supervision from the layout restriction and semantic coherence requirement and exploit it to guide the image synthesis process. Moreover, to improve the physical coherence, we develop a Self-similarity Coherence Attention (SCA) module to explicitly integrate local contextual physical coherence relation into each pixel's generation process. Specifically, we adopt a self-similarity map to encode the physical coherence restrictions and employ it to extract coherent features from text embedding. Through visualization of our self-similarity map, we explore the essence of SCA, revealing that its effectiveness is not only in capturing reliable physical coherence patterns but also in enhancing complex texture generation. Extensive experiments demonstrate the superiority of our proposed method.
RSGen: Enhancing Layout-Driven Remote Sensing Image Generation with Diverse Edge Guidance
School of Future Science and Engineering, Soochow University, Suzhou, China（未来科学与工程学院，苏州大学）；D-Robotics, Beijing, China（北京D-机器人）；Fraunhofer Institute for Applied Information Technology, Sankt Augustin, Germany（弗劳恩霍夫应用信息技术研究所）；School of Mathematical and Computing Sciences, Heriot-Watt University Malaysia, Putrajaya, Malaysia（数学与计算科学学院，赫瑞-瓦德大学马来西亚分校）；Institute of Automation, Chinese Academy of Sciences, Beijing, China（自动化研究所，中国科学院北京分院）
Enhancing Retinal Vessel Segmentation Generalization via Layout-Aware Generative Modelling
Faculty of Biomedical Engineering, Technion-IIT（以色列理工学院生物医学工程学院）；Faculty of Mathematics, Technion-IIT（以色列理工学院数学学院）；KU Leuven（鲁汶大学）；UZ Leuven（鲁汶大学医院）；São Paulo Federal University（圣保罗联邦大学）；Massachusetts Institute of Technology（麻省理工学院）；School of Electrical and Computer Engineering, Ben-Gurion University of the Negev（本-古里安大学内盖夫分校电气与计算机工程学院）
EchoGen: Cycle-Consistent Learning for Unified Layout-Image Generation and Understanding
School of Cyber Science and Technology, University of Science and Technology of China（中国科学技术大学信息科学与技术学院）；Tongji University（同济大学）；Sun Yat-sen University（中山大学）；Fudan University（复旦大学）；Hefei University of Technology（合肥工业大学）；Anhui Province Key Laboratory of Digital Security（安徽省数字安全重点实验室）
Foreground Focus: Enhancing Coherence and Fidelity in Camouflaged Image Generation
Institute of Information Science, Academia Sinica（中央研究院资讯科学研究所）；National Yang Ming Chiao Tung University（国立阳明交通大学）；Jilin University（吉林大学）；MediaTek（联发科）；National Taiwan University（国立台湾大学）
Graph-PiT: Enhancing Structural Coherence in Part-Based Image Synthesis via Graph Priors
Guangdong Provincial Key Laboratory of Ultra High Definition Immersive Media Technology（广东省超高清沉浸式媒体技术重点实验室）；Shenzhen Graduate School, Peking University, China（北京大学深圳研究生院）；Department of Computer Vision, Mohamed bin Zayed University of Artificial Intelligence, Abu Dhabi, United Arab Emirates（穆罕默德·本·扎耶德人工智能大学计算机视觉系）
The Chinese University of Hong Kong（香港中文大学）；University of Science and Technology of China（中国科学技术大学）；SenseTime Research（商汤科技研究院）
Semantic image synthesis aims at generating photorealistic images from semantic layouts. Previous approaches with conditional generative adversarial networks (GAN) show state-of-the-art performance on this task, which either feed the semantic label maps as inputs to the generator, or use them to modulate the activations in normalization layers via affine transformations. We argue that convolutional kernels in the generator should be aware of the distinct semantic labels at different locations when generating images. In order to better exploit the semantic layout for the image generator, we pro…
We present an approach to enhancing the realism of synthetic images. The images are enhanced by a convolutional network that leverages intermediate representations produced by conventional rendering pipelines. The network is trained via a novel adversarial objective, which provides strong supervision at multiple perceptual levels. We analyze scene layout distributions in commonly used datasets and find that they differ in important ways. We hypothesize that this is one of the causes of strong artifacts that can be observed in the results of many prior methods. To address this we propose a new …
FICGen: Frequency-Inspired Contextual Disentanglement for Layout-driven Degraded Image Generation
State Key Laboratory of Virtual Reality Technology and Systems, SCSE&QRI, Beihang University（虚拟现实技术与系统国家重点实验室，北京航空航天大学）；Geely Automobile Research Institute (Ningbo) Co., Ltd（吉利汽车研究院（宁波）有限公司）
Enhancing Low-Cost Video Editing with Lightweight Adaptors and Temporal-Aware Inversion
University of Minnesota—Twin Cities（明尼苏达大学-双城分校）；Peking University（北京大学）；University of Electronic Science and Technology of China（电子科技大学）；Xiamen University（厦门大学）；Emory University（埃默里大学）；Washington University, Saint Louis（圣路易斯华盛顿大学）；Cornell University（康奈尔大学）；Nanjing University（南京大学）；Santa Clara University（圣克拉拉大学）；Tsinghua University（清华大学）；ByteDance Inc.（字节跳动公司）
Doc-CoB: Enhancing Document Understanding with Visual Chain-of-Boxes Reasoning
Zhejiang Key Laboratory of Accessible Perception and Intelligent Systems, Zhejiang University（浙江可感知与智能系统重点实验室，浙江大学）；Alibaba Group（阿里巴巴集团）；Shanghai AI Laboratory（上海人工智能实验室）
Enhancing License Plate Super-Resolution: A Layout-Aware and Character-Driven Approach
Federal University of Paraná（巴拉那联邦大学）；Pontifical Catholic University of Paraná（巴拉那天主教大学）；Brazilian Federal Police（巴西联邦警察）；Federal University of Minas Gerais（米纳斯吉拉斯联邦大学）
RealCamo: Boosting Real Camouflage Synthesis with Layout Controls and Textual-Visual Guidance
College of Artificial Intelligence, Nankai University, Tianjin, China（人工智能学院，南开大学，天津，中国）；School of Data Science, Fudan University, Shanghai, China（数据科学学院，复旦大学，上海，中国）
