用于 3D GAN 反演的元辅助网络
Meta-Auxiliary Network for 3D GAN Inversion
AI总结：
提出一种基于3D GAN的元辅助框架，通过两阶段策略结合辅助网络与元学习，在保证快速推理的同时实现高重建质量与多视图一致性，提升了反演与编辑任务性能。
摘要：
近年来，真实世界图像处理取得了惊人的进展。GAN inversion（GAN 反演）旨在将真实图像忠实地映射到潜空间编码，是该流程的第一步。然而，现有的 GAN inversion 方法无法同时实现高重建质量和快速推理。此外，现有方法基于 2D GAN 构建，缺乏强制多视图一致性的显式机制。在本工作中，我们提出了一种新颖的元辅助框架，同时利用新开发的 3D GAN 作为生成器。所提出的方法采用两阶段策略。在第一阶段，我们使用现成的反演技术将输入图像反演为可编辑的潜空间编码。提出的辅助网络以给定图像为输入来细化生成器参数，同时预测卷积层权重的偏移量和体渲染的采样位置。在第二阶段，我们执行元学习以使辅助网络快速适应输入图像，然后通过元学习的辅助网络合成最终重建图像。大量实验表明，我们的方法在反演和编辑任务上均取得了更好的性能。
英文摘要：
Real-world image manipulation has achieved fantastic progress in recent years. GAN inversion, which aims to map the real image to the latent code faithfully, is the first step in this pipeline. However, existing GAN inversion methods fail to achieve high reconstruction quality and fast inference at the same time. In addition, existing methods are built on 2D GANs and lack explicitly mechanisms to enforce multi-view consistency.In this work, we present a novel meta-auxiliary framework, while leveraging the newly developed 3D GANs as generator. The proposed method adopts a two-stage strategy. In the first stage, we invert the input image to an editable latent code using off-the-shelf inversion techniques. The auxiliary network is proposed to refine the generator parameters with the given image as input, which both predicts offsets for weights of convolutional layers and sampling positions of volume rendering. In the second stage, we perform meta-learning to fast adapt the auxiliary network to the input image, then the final reconstructed image is synthesized via the meta-learned auxiliary network. Extensive experiments show that our method achieves better performances on both inversion and editing tasks.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
