We introduce MyStyle, a personalized deep generative prior trained with a few shots of an individual. MyStyle allows to reconstruct, enhance and edit images of a specific person, such that the output is faithful to the person's key facial characteristics. Given a small reference set of portrait images of a person (~100), we tune the weights of a pretrained StyleGAN face generator to form a local, low-dimensional, personalized manifold in the latent space. We show that this manifold constitutes a personalized region that spans latent codes associated with diverse portrait images of the individual. Moreover, we demonstrate that we obtain a personalized generative prior, and propose a unified approach to apply it to various ill-posed image enhancement problems, such as inpainting and super-resolution, as well as semantic editing. Using the personalized generative prior we obtain outputs that exhibit high-fidelity to the input images and are also faithful to the key facial characteristics of the individual in the reference set. We demonstrate our method with fair-use images of numerous widely recognizable individuals for whom we have the prior knowledge for a qualitative evaluation of the expected outcome. We evaluate our approach against few-shots baselines and show that our personalized prior, quantitatively and qualitatively, outperforms state-of-the-art alternatives.
UVMap-ID: A Controllable and Personalized UV Map Generative Model
University of Trento(特伦托大学);Ocean University of China(中国海洋大学);ETH Zürich(苏黎世联邦理工学院);PicsArt AI Research(PicsArt人工智能研究院);Georgia Institute of Technology(佐治亚理工学院);Fondazione Bruno Kessler(布鲁诺·凯斯勒基金会)
本文提出 UVMap-ID,通过微调集成人脸融合模块的文本到图像扩散模型,并利用带文本与 Face ID 标注的小规模属性均衡数据集,实现可控、个性化的人物 UV Map 生成与多维质量评估。
Fundus2Globe: Generative AI-Driven 3D Digital Twins for Personalized Myopia Management
School of Optometry, The Hong Kong Polytechnic University(香港理工大学验光学院);Swiss Federal Institute of Technology Lausanne (EPFL)(瑞士洛桑联邦理工学院);Centre for Eye and Vision Research (CEVR)(眼与视觉研究中心)
'A net for everyone': fully personalized and unsupervised neural networks trained with longitudinal data from a single patient
German Cancer Research Center (DKFZ)(德国癌症研究中心);Heidelberg University(海德堡大学);University Hospital Essen (AöR)(埃森大学医院);University Medicine Essen(埃森大学医学院);German Cancer Consortium (DKTK)(德国癌症联盟)
DreamVTON: Customizing 3D Virtual Try-on with Personalized Diffusion Models
Shenzhen Campus of Sun Yat-sen University(中山大学深圳校区);ByteDance(字节跳动);Carnegie Mellon University(卡内基梅隆大学);Research Institute of Multiple Agents and Embodied Intelligence(多智能体与具身智能研究院);Peng Cheng Laboratory(鹏城实验室)
本文提出 DreamVTON 模型,通过结合个性化扩散模型、Densepose-guided ControlNet 和基于模板的优化机制,分别优化 3D 人体的几何与纹理,解决了个性化模块削弱多视图合成能力的问题,实现了高质量的定制化 3D 虚拟试穿。
DreamShot: Personalized Storyboard Synthesis with Video Diffusion Prior
Sun Yat-sen University(中山大学);Peng Cheng Laboratory(鹏城实验室);ByteDance Intelligent Creation(字节跳动智能创作);Guangdong Key Laboratory of Big Data Analysis and Processing(广东省大数据分析与处理重点实验室)