Face inpainting techniques recover missing or occluded facial regions in a visually realistic manner, but preserving the identity in the final output remains a fundamental challenge. Identity consistency is crucial for downstream applications such as face recognition, digital forensics, and human-computer interaction, where even subtle identity distortions can significantly degrade performance or trust. Although diffusion-based generative models have recently achieved remarkable progress in image inpainting, they often struggle to faithfully retain individual-specific facial characteristics. On the other hand, existing identity-aware methods typically rely on costly fine-tuning, auxiliary supervision, or exhibit limited robustness to diverse occlusions, poses, and facial variations. To address these limitations, we propose ID-ControlNet, an identity-preserving face inpainting framework built upon latent diffusion models. Based on ControlNet architecture, our approach conditions the diffusion process on facial identity embeddings extracted from a pretrained face recognition network. This design enables reconstruction of occluded facial regions while maintaining global facial coherence and identity fidelity. Furthermore, we introduce an identity consistency and triplet loss training strategy that explicitly enforces alignment between the generated face and the target identity representation. Extensive experiments on CelebA-HQ, FFHQ, and on a new E-Mask dataset demonstrate that ID-ControlNet significantly improves identity preservation over standard diffusion-based inpainting methods, achieving performance comparable to SOTA identity-aware approaches.
DreamMix: Decoupling Object Attributes for Enhanced Editability in Customized Image Inpainting
Information and Communication Engineering（信息与通信工程）；Dalian University of Technology（大连理工大学）；ZMO.AI Inc.（ZMO.AI公司）；University of Electronic Science and Technology of China（电子科技大学）
Manchester Metropolitan University（曼彻斯特城市大学）；Image Metrics Ltd（Image Metrics有限公司）
Image inpainting is a key technique in image processing task to predict the missing regions and generate realistic images. Given the advancement of existing generative inpainting models with feature extraction, propagation and reconstruction capabilities, there is lack of high-quality feature extraction and transfer mechanisms in deeper layers to tackle persistent aberrations on the generated inpainted regions. Our method, V-LinkNet, develops high-level feature transference to deep level textural context of inpainted regions our work, proposes a novel technique of combining encoders learning t…
Institute of Software, Chinese Academy of Sciences（中国科学院软件研究所）；University of Chinese Academy of Sciences（中国科学院大学）；Nanjing Institute of Software Technology（南京软件技术研究院）；University of North Texas（北得克萨斯大学）
Image inpainting seeks a semantically consistent way to recover the corrupted image in the light of its unmasked content. Previous approaches usually reuse the well-trained GAN as effective prior to generate realistic patches for missing holes with GAN inversion. Nevertheless, the ignorance of a hard constraint in these algorithms may yield the gap between GAN inversion and image inpainting. Addressing this problem, in this paper, we devise a novel GAN inversion model for image inpainting, dubbed InvertFill, mainly consisting of an encoder with a pre-modulation module and a GAN generator with …
WarpGAN: Warping-Guided 3D GAN Inversion with Style-Based Novel View Inpainting
Key Laboratory of Multimedia Trusted Perception and Efficient Computing, Ministry of Education of China, Xiamen University, P.R. China（中国教育部多媒体可信感知与高效计算重点实验室，厦门大学）；Department of Statistical Science, University College London, UK（伦敦大学学院统计学系）
Malong Technologies（码隆科技）；Shenzhen Malong Artificial Intelligence Research Center（深圳码隆人工智能研究中心）；University of Maryland, College Park（马里兰大学帕克分校）
Visual compatibility is critical for fashion analysis, yet is missing in existing fashion image synthesis systems. In this paper, we propose to explicitly model visual compatibility through fashion image inpainting. To this end, we present Fashion Inpainting Networks (FiNet), a two-stage image-to-image generation framework that is able to perform compatible and diverse inpainting. Disentangling the generation of shape and appearance to ensure photorealistic results, our framework consists of a shape generation network and an appearance generation network. More importantly, for each generation …
Training-Free Inpainting Across Domains with a Frozen Text-to-Image Diffusion Model
School of Data Science, The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳）数据科学学院）；School of Science and Engineering, The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳）理工学院）
Indian Institute of Technology Kharagpur（印度理工学院卡拉格普尔分校）
Contemporary benchmark methods for image inpainting are based on deep generative models and specifically leverage adversarial loss for yielding realistic reconstructions. However, these models cannot be directly applied on image/video sequences because of an intrinsic drawback- the reconstructions might be independently realistic, but, when visualized as a sequence, often lacks fidelity to the original uncorrupted sequence. The fundamental reason is that these methods try to find the best matching latent space representation near to natural image manifold without any explicit distance based lo…
Image inpainting is the task of filling-in missing regions of a damaged or incomplete image. In this work we tackle this problem not only by using the available visual data but also by incorporating image semantics through the use of generative models. Our contribution is twofold: First, we learn a data latent space by training an improved version of the Wasserstein generative adversarial network, for which we incorporate a new generator and discriminator architecture. Second, the learned semantic information is combined with a new optimization loss for inpainting whose minimization infers the…
University of Wisconsin-Madison（威斯康星大学麦迪逊分校）；Adobe Research（奥多比研究院）
We introduce a new method for diverse foreground generation with explicit control over various factors. Existing image inpainting based foreground generation methods often struggle to generate diverse results and rarely allow users to explicitly control specific factors of variation (e.g., varying the facial identity or expression for face inpainting results). We leverage contrastive learning with latent codes to generate diverse foreground results for the same masked input. Specifically, we define two sets of latent codes, where one controls a pre-defined factor (``known''), and the other con…
University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）
Semantic image inpainting is a challenging task where large missing regions have to be filled based on the available visual data. Existing methods which extract information from only a single image generally produce unsatisfactory results due to the lack of high level context. In this paper, we propose a novel method for semantic image inpainting, which generates the missing content by conditioning on the available data. Given a trained generative model, we search for the closest encoding of the corrupted image in the latent image manifold using our context and prior losses. This encoding is t…
Controllable Localized Face Anonymization Via Diffusion Inpainting
Center for Machine Vision and Signal Analysis (CMVS)（机器视觉与信号分析中心）；University of Oulu（奥卢大学）；ELLIS Institute Finland（芬兰ELLIS研究所）；The Machine Learning Group（机器学习小组）；UiT The Arctic University of Norway（挪威北冰洋大学）
SuperInpaint: Learning Detail-Enhanced Attentional Implicit Representation for Super-resolutional Image Inpainting
University of South Carolina（南卡罗来纳大学）；Hong Kong Baptist University（香港浸会大学）；Cleveland State University（克利夫兰州立大学）；Center for Frontier AI Research (CFAR), A*STAR（新加坡科技研究局前沿人工智能研究中心（CFAR））
DreamVideo-Omni: Omni-Motion Controlled Multi-Subject Video Customization with Latent Identity Reinforcement Learning
Fudan University（复旦大学）；The Hong Kong University of Science and Technology（香港科技大学）；Tongyi Lab, Alibaba Group（阿里云实验室）；Zhejiang University（浙江大学）；MMLab, The University of Hong Kong（香港大学MMLab）；Nanyang Technological University（南洋理工大学）；Show Lab, National University of Singapore（新加坡国立大学Show Lab）
DiffKAN-Inpainting: KAN-based Diffusion model for brain tumor inpainting
School of Biomedical Engineering, ShanghaiTech University（上海科技大学生物医学工程学院）；The Alan Turing Institute（阿兰·图灵研究所）；School of Engineering, University of Birmingham（伯明翰大学工程学院）；College of Engineering and Physical Sciences, University of Birmingham（伯明翰大学工程与物理科学学院）
Beyond Inference Intervention: Identity-Decoupled Diffusion for Face Anonymization
School of Computer Science and Engineering, South China University of Technology（华南理工大学计算机科学与工程学院）；School of Future Technology, South China University of Technology（华南理工大学未来技术学院）；School of Computing and Information Systems, Singapore Management University（新加坡管理学院计算机与信息学院）
Télécom Paris, Institut Polytechnique de Paris（巴黎高等电信学院，巴黎综合理工学院）；InterDigital R&I（InterDigital研究与创新部门）
High quality facial image editing is a challenging problem in the movie post-production industry, requiring a high degree of control and identity preservation. Previous works that attempt to tackle this problem may suffer from the entanglement of facial attributes and the loss of the person's identity. Furthermore, many algorithms are limited to a certain task. To tackle these limitations, we propose to edit facial attributes via the latent space of a StyleGAN generator, by training a dedicated latent transformation network and incorporating explicit disentanglement and identity preservation t…
