Controllable Localized Face Anonymization Via Diffusion Inpainting
Center for Machine Vision and Signal Analysis (CMVS)（机器视觉与信号分析中心）；University of Oulu（奥卢大学）；ELLIS Institute Finland（芬兰ELLIS研究所）；The Machine Learning Group（机器学习小组）；UiT The Arctic University of Norway（挪威北冰洋大学）
The growing use of portrait images in computer vision highlights the need to protect personal identities. At the same time, anonymized images must remain useful for downstream computer vision tasks. In this work, we propose a unified framework that leverages the inpainting ability of latent diffusion models to generate realistic anonymized images. Unlike prior approaches, we have complete control over the anonymization process by designing an adaptive attribute-guidance module that applies gradient correction during the reverse denoising process, aligning the facial attributes of the generated image with those of the synthesized target image. Our framework also supports localized anonymization, allowing users to specify which facial regions are left unchanged. Extensive experiments conducted on the public CelebA-HQ and FFHQ datasets show that our method outperforms state-of-the-art approaches while requiring no additional model training. The source code is available on our page.
Audi AG（奥迪股份公司）；Technische Hochschule Ingolstadt（因戈尔施塔特应用技术大学）
This work tackles the problem of temporally coherent face anonymization in natural video streams.We propose JaGAN, a two-stage system starting with detecting and masking out faces with black image patches in all individual frames of the video. The second stage leverages a privacy-preserving Video Generative Adversarial Network designed to inpaint the missing image patches with artificially generated faces. Our initial experiments reveal that image based generative models are not capable of inpainting patches showing temporal coherent appearance across neighboring video frames. To address this …
Beyond Inference Intervention: Identity-Decoupled Diffusion for Face Anonymization
School of Computer Science and Engineering, South China University of Technology（华南理工大学计算机科学与工程学院）；School of Future Technology, South China University of Technology（华南理工大学未来技术学院）；School of Computing and Information Systems, Singapore Management University（新加坡管理学院计算机与信息学院）
CRIPAC & NLPR, CASIA（中国科学院自动化研究所模式识别国家重点实验室与智能感知与计算研究中心）；School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）
De-identification of face data has drawn increasing attention in recent years. It is important to protect people's identities meanwhile keeping the utility of the data in many computer vision tasks. We propose a Controllable Face Anonymization Network (CFA-Net), a novel approach that can anonymize the identity of given faces in images and videos, based on a generator that can disentangle face identity from other image contents. We reach the goal of controllable face anonymization through manipulating identity vectors in the generator's identity representation space. Various anonymized faces de…
Improving Text-guided Object Inpainting with Semantic Pre-inpainting
School of Computer Science, Fudan University（复旦大学计算机科学学院）；HiDream.ai Inc.（HiDream.ai公司）；Shanghai Collaborative Innovation Center of Intelligent Visual Computing（上海智能视觉计算协同创新中心）
University of Science and Technology of China（中国科学技术大学）
Generative adversarial nets (GANs) have been successfully applied in many fields like image generation, inpainting, super-resolution and drug discovery, etc., by now, the inner process of GANs is far from been understood. To get deeper insight of the intrinsic mechanism of GANs, in this paper, a method for interpreting the latent space of GANs by analyzing the correlation between latent variables and the corresponding semantic contents in generated images is proposed. Unlike previous methods that focus on dissecting models via feature visualization, the emphasis of this work is put on the vari…
Key Laboratory of Ministry of Education for Image Processing and Intelligent Control, School of Artificial Intelligence and Automation, Huazhong University of Science and Technology（教育部图像处理与智能控制重点实验室，人工智能与自动化学院，华中科技大学）；Alibaba Group（阿里巴巴集团）；Nanyang Technological University（南洋理工大学）；Nanjing University（南京大学）；Zhejiang University（浙江大学）
本文提出 ReplaceAnyone 框架，将局部人物替换与插入任务形式化为姿态引导的图像条件视频修复，通过多样化掩码、增强视觉引导、混合修复编码器和两阶段优化，在保留背景与运动的同时实现无缝人物替换，并兼容 3D-UNet 和 DiT 视频模型。
Transformer based Pluralistic Image Completion with Reduced Information Loss
School of Computer Science and Technology, Beijing Institute of Technology（北京理工大学计算机科学与技术学院）；School of Information Science and Technology, University of Science and Technology of China（中国科学技术大学信息科学技术学院）；Microsoft Cloud AI（微软云人工智能部门）；Wormpex AI Research LLC（Wormpex人工智能研究有限责任公司）
LeFusion: Controllable Pathology Synthesis via Lesion-Focused Diffusion Models
University of Science and Technology of China (USTC)（中国科学技术大学）；Beihang University（北京航空航天大学）；Swiss Federal Institute of Technology Lausanne (EPFL)（洛桑联邦理工学院）；Stanford University（斯坦福大学）
Center for Research on Intelligent Perception and Computing, NLPR, CASIA（中国科学院自动化研究所模式识别国家重点实验室智能感知与计算研究中心）；University of Chinese Academy of Sciences（中国科学院大学）；Meituandianping Group（美团点评集团）；Artificial Intelligence Research, CAS（中国科学院人工智能研究院）；Center for Excellence in Brain Science and Intelligence Technology, CAS（中国科学院脑科学与智能技术卓越创新中心）
Face portrait editing has achieved great progress in recent years. However, previous methods either 1) operate on pre-defined face attributes, lacking the flexibility of controlling shapes of high-level semantic facial components (e.g., eyes, nose, mouth), or 2) take manually edited mask or sketch as an intermediate representation for observable changes, but such additional input usually requires extra efforts to obtain. To break the limitations (e.g. shape, mask or sketch) of the existing methods, we propose a novel framework termed r-FACE (Reference-guided FAce Component Editing) for diverse…
Weizmann Institute of Science（魏茨曼科学研究所）；Google Research（谷歌研究院）
We present a novel GAN-based model that utilizes the space of deep features learned by a pre-trained classification model. Inspired by classical image pyramid representations, we construct our model as a Semantic Generation Pyramid -- a hierarchical framework which leverages the continuum of semantic information encapsulated in such deep features; this ranges from low level information contained in fine features to high level, semantic information contained in deeper features. More specifically, given a set of features extracted from a reference image, our model generates diverse image samples…
Fudan University（复旦大学）；China University of Petroleum（中国石油大学）
Face inpainting aims at plausibly predicting missing pixels of face images within a corrupted region. Most existing methods rely on generative models learning a face image distribution from a big dataset, which produces uncontrollable results, especially with large-scale missing regions. To introduce strong control for face inpainting, we propose a novel reference-guided face inpainting method that fills the large-scale missing region with identity and texture control guided by a reference face image. However, generating high-quality results under imposing two control signals is challenging. T…
Controllable Diffusion-Based Lesion Inpainting for Scalable Histopathology Data Augmentation
Department of Diagnostic Radiology, Li Ka Shing Faculty of Medicine, The University of Hong Kong（香港大学李嘉诚医学院诊断放射学系）；Department of Pathology, Li Ka Shing Faculty of Medicine, The University of Hong Kong（香港大学李嘉诚医学院病理学系）；Department of Anatomical and Cellular Pathology, Prince of Wales Hospital, The Chinese University of Hong Kong（香港中文大学威尔斯亲王医院解剖及细胞病理学系）；Department of Infectious Diseases and Public Health, Jockey Club College of Veterinary Medicine and Life Sciences, City University of Hong Kong（香港城市大学赛马会动物医学及生命科学院传染病及公共卫生学系）；CityU Veterinary Diagnostic Laboratory Co., Ltd., City University of Hong Kong（香港城市大学城市大学兽医诊断实验室有限公司）
HHMR: Holistic Hand Mesh Recovery by Enhancing the Multimodal Controllability of Graph Diffusion Models
Tsinghua University（清华大学）；Beijing Normal University（北京师范大学）；Beijing National Research Center for Information Science and Technology, Tsinghua University（北京信息科学与技术国家研究中心（清华大学））
Stable-Hair v2: Real-World Hair Transfer via Multiple-View Diffusion Model
Institute of Information Science, Beijing Jiaotong University（北京交通大学信息科学学院）；School of Computer Science, Ocean University of China（中国海洋大学计算机学院）；Department of Computer Science and Engineering, The Chinese University of Hong Kong（香港中文大学计算机科学与工程系）；Artificial Intelligence department, Tiamat AI（Tiamat AI人工智能部门）；Department of Information Engineering and Computer Science (DISI), University of Trento（特伦托大学信息工程与计算机科学系）
From Redaction to Restoration: Deep Learning for Medical Image Anonymization and Reconstruction
Center for Artificial Intelligence, BCVI, Northwestern Medicine（西北医学BCVI人工智能中心）；Department of Electrical and Computer Engineering, Northwestern University（西北大学电气与计算机工程系）；Department of Surgery, Northwestern University（西北大学外科学系）；Xtasis Inc.（Xtasis公司）；Medtronic（美敦力公司）；Departments of Radiology and Clinical Neurosciences, University of Calgary（卡尔加里大学放射学与临床神经科学系）
