This paper describes a new model which generates images in novel poses e.g. by altering face expression and orientation, from just a few instances of a human subject. Unlike previous approaches which require large datasets of a specific person for training, our approach may start from a scarce set of images, even from a single image. To this end, we introduce Character Adaptive Identity Normalization GAN (CainGAN) which uses spatial characteristic features extracted by an embedder and combined across source images. The identity information is propagated throughout the network by applying conditional normalization. After extensive adversarial training, CainGAN receives figures of faces from a certain individual and produces new ones while preserving the person's identity. Experimental results show that the quality of generated images scales with the size of the input set used during inference. Furthermore, quantitative measurements indicate that CainGAN performs better compared to other methods when training data is limited.
In this paper, we present an approach based on reinforcement learning for eye tracking data manipulation. It is based on two opposing agents, where one tries to classify the data correctly and the second agent looks for patterns in the data, which get manipulated to hide specific information. We show that our approach is successfully applicable to preserve the privacy of the subjects. For this purpose, we evaluate our approach iteratively to showcase the behavior of the reinforcement learning based approach. In addition, we evaluate the importance of temporal, as well as spatial, information o…
CRIPAC & NLPR, CASIA（中国科学院自动化研究所模式识别国家重点实验室与智能感知与计算研究中心）；School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）
De-identification of face data has drawn increasing attention in recent years. It is important to protect people's identities meanwhile keeping the utility of the data in many computer vision tasks. We propose a Controllable Face Anonymization Network (CFA-Net), a novel approach that can anonymize the identity of given faces in images and videos, based on a generator that can disentangle face identity from other image contents. We reach the goal of controllable face anonymization through manipulating identity vectors in the generator's identity representation space. Various anonymized faces de…
Technical University of Munich（慕尼黑工业大学）；Munich Center for Machine Learning（慕尼黑机器学习中心）；Huawei Noah’s Ark Lab（华为诺亚方舟实验室）；Google（谷歌）；Johns Hopkins University（约翰斯·霍普金斯大学）
Graph representation of objects and their relations in a scene, known as a scene graph, provides a precise and discernible interface to manipulate a scene by modifying the nodes or the edges in the graph. Although existing works have shown promising results in modifying the placement and pose of objects, scene manipulation often leads to losing some visual characteristics like the appearance or identity of objects. In this work, we propose DisPositioNet, a model that learns a disentangled representation for each object for the task of image manipulation using scene graphs in a self-supervised …
We present a generative autoencoder that provides fast encoding, faithful reconstructions (eg. retaining the identity of a face), sharp generated/reconstructed samples in high resolutions, and a well-structured latent space that supports semantic manipulation of the inputs. There are no current autoencoder or GAN models that satisfactorily achieve all of these. We build on the progressively growing autoencoder model PIONEER, for which we completely alter the training dynamics based on a careful analysis of recently introduced normalization schemes. We show significantly improved visual and qua…
Camera sensors are increasingly being combined with machine learning to perform various tasks such as intelligent surveillance. Due to its computational complexity, most of these machine learning algorithms are offloaded to the cloud for processing. However, users are increasingly concerned about privacy issues such as function creep and malicious usage by third-party cloud providers. To alleviate this, we propose an edge-based filtering stage that removes privacy-sensitive attributes before the sensor data are transmitted to the cloud. We use state-of-the-art image manipulation techniques tha…
AU-Blendshape for Fine-grained Stylized 3D Facial Expression Manipulation
Beihang University（北京航空航天大学）；Peng Cheng Laboratory（鹏城实验室）；North China University of Technology（华北理工大学）；University of Washington（华盛顿大学）；University of Copenhagen（哥本哈根大学）
University of Southern California（南加利福尼亚大学）；Emory University（埃默里大学）
Previous methods have dealt with discrete manipulation of facial attributes such as smile, sad, angry, surprise etc, out of canonical expressions and they are not scalable, operating in single modality. In this paper, we propose a novel framework that supports continuous edits and multi-modality portrait manipulation using adversarial learning. Specifically, we adapt cycle-consistency into the conditional setting by leveraging additional facial landmarks information. This has two effects: first cycle mapping induces bidirectional manipulation and identity preserving; second pairing samples fro…
When the Edit Changes the Patient: Measuring Identity Preservation in Counterfactual Retinal Images
Technical University of Munich（慕尼黑工业大学）；TUM University Hospital（慕尼黑工业大学医院）；Munich Center of Machine Learning (MCML)（慕尼黑机器学习中心）；Munich Data Science Institute (MDSI)（慕尼黑数据科学研究所）；Sapienza University of Rome（罗马第一大学）；Faculty of Medicine, University of Southampton（南安普顿大学医学院）；Moorfields Eye Hospital NHS Foundation Trust（摩尔菲尔兹眼科医院NHS信托基金）；Imperial College London（伦敦帝国学院）
Few-Shot Generative Model Adaption via Identity Injection and Preservation
School of Cyberspace Security, Hangzhou Dianzi University（杭州电子科技大学网络安全学院）；Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）；School of Software Engineering, Xi’an Jiaotong University（西安交通大学软件工程学院）；Lishui University and Lishui Institute of Hangzhou Dianzi University（丽水大学和杭州电子科技大学丽水研究所）；School of Faculty of Applied Science, Macao Polytechnic University（澳门理工学院应用科学学院）；School of Communication Engineering, Hangzhou Dianzi University（杭州电子科技大学通信工程学院）
Realistic age-progressed photos provide invaluable biometric information in a wide range of applications. In recent years, deep learning-based approaches have made remarkable progress in modeling the aging process of the human face. Nevertheless, it remains a challenging task to generate accurate age-progressed faces from infant or toddler photos. In particular, the lack of visually detectable gender characteristics and the drastic appearance changes in early life contribute to the difficulty of the task. We propose a new deep learning method inspired by the successful Conditional Adversarial …
Training-Free Diffusion Framework for Stylized Image Generation with Identity Preservation
Computer Vision Group, Part AI Research Center（部分人工智能研究中心计算机视觉组）；Department of Computer Engineering, Amirkabir University of Technology（阿米卡比尔技术大学计算机工程系）
Nowadays, the wide application of virtual digital human promotes the comprehensive prosperity and development of digital culture supported by digital economy. The personalized portrait automatically generated by AI technology needs both the natural artistic style and human sentiment. In this paper, we propose a novel StyleIdentityGAN model, which can ensure the identity and artistry of the generated portrait at the same time. Specifically, the style-enhanced module focuses on artistic style features decoupling and transferring to improve the artistry of generated virtual face images. Meanwhile…
We present a novel framework to generate images of different age while preserving identity information, which is known as face aging. Different from most recent popular face aging networks utilizing Generative Adversarial Networks(GANs) application, our approach do not simply transfer a young face to an old one. Instead, we employ the edge map as intermediate representations, firstly edge maps of young faces are extracted, a CycleGAN-based network is adopted to transfer them into edge maps of old faces, then another pix2pixHD-based network is adopted to transfer the synthesized edge maps, conc…
University of Pisa（比萨大学）；University of Trento（特伦托大学）；Fondazione Bruno Kessler（布鲁诺·凯塞勒基金会）；ETH Zürich（苏黎世联邦理工学院）
We propose AnonyGAN, a GAN-based solution for face anonymisation which replaces the visual information corresponding to a source identity with a condition identity provided as any single image. With the goal to maintain the geometric attributes of the source face, i.e., the facial pose and expression, and to promote more natural face generation, we propose to exploit a Bipartite Graph to explicitly model the relations between the facial landmarks of the source identity and the ones of the condition identity through a deep model. We further propose a landmark attention model to relax the manual…
Iran University of Science and Technology（伊朗科技大学）；Amirkabir University of Technology（阿米尔卡比尔理工大学）
The use of social media websites and applications has become very popular and people share their photos on these networks. Automatic recognition and tagging of people's photos on these networks has raised privacy preservation issues and users seek methods for hiding their identities from these algorithms. Generative adversarial networks (GANs) are shown to be very powerful in generating face images in high diversity and also in editing face images. In this paper, we propose a Generative Mask-guided Face Image Manipulation (GMFIM) model based on GANs to apply imperceptible editing to the input …
When there is a mismatch between the target identity and the driver identity, face reenactment suffers severe degradation in the quality of the result, especially in a few-shot setting. The identity preservation problem, where the model loses the detailed information of the target leading to a defective output, is the most common failure mode. The problem has several potential sources such as the identity of the driver leaking due to the identity mismatch, or dealing with unseen large poses. To overcome such problems, we introduce components that address the mentioned problem: image attention …
Diffusion-based Adversarial Identity Manipulation for Facial Privacy Protection
School of Computer Science；Engineering, Sun Yat-sen University Guangzhou China；State Key Laboratory of Mathematical Engineering；Engineering, Sun Yat-sen University
Samsung SDS（三星SDS）；Seoul National University（首尔大学）；POSTECH（浦项工科大学）
In this work, we present Facial Identity Controllable GAN (FICGAN) for not only generating high-quality de-identified face images with ensured privacy protection, but also detailed controllability on attribute preservation for enhanced data utility. We tackle the less-explored yet desired functionality in face de-identification based on the two factors. First, we focus on the challenging issue to obtain a high level of privacy protection in the de-identification task while uncompromising the image quality. Second, we analyze the facial attributes related to identity and non-identity and explor…
Identity Preserving 3D Head Stylization with Multiview Score Distillation
Bilkent University（比尔肯大学）；ETH Zurich（苏黎世联邦理工学院）；Max Planck Institute（马克斯·普朗克研究所）；INSAIT, Sofia University “St. Kliment Ohridski”（INSAIT，索菲亚大学“圣克莱门特·奥赫里茨基”）
