arXiv 2511.081782025-11-11cs.CV

WarpGAN：基于变形引导的3D GAN逆变换与风格化新视角修复

WarpGAN: Warping-Guided 3D GAN Inversion with Style-Based Novel View Inpainting

Key Laboratory of Multimedia Trusted Perception and Efficient Computing, Ministry of Education of China, Xiamen University, P.R. China（中国教育部多媒体可信感知与高效计算重点实验室，厦门大学）；Department of Statistical Science, University College London, UK（伦敦大学学院统计学系）

Kaitao Huang, Yan Yan, Jing-Hao Xue, Hanzi Wang

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

针对3D GAN逆变换中遮挡区域因潜在代码信息损失导致质量差的问题，提出WarpGAN，通过变形修复策略结合图像修复，利用SVINet处理遮挡区域，实现更优的单镜头新视角合成。

摘要：

3D GAN逆变换将单张图像投影到预训练3D GAN的潜在空间中，以实现单镜头新视角合成，这要求可见区域具有高保真度，遮挡区域具有真实感和多视角一致性。然而，现有方法侧重于可见区域的重建，而遮挡区域的生成仅依赖于3D GAN的生成先验。由于低比特率潜在代码导致的信息损失，生成的遮挡区域质量往往较差。为解决这一问题，我们引入变形修复策略，将图像修复融入3D GAN逆变换，提出一种新的3D GAN逆变换方法WarpGAN。具体而言，我们首先使用3D GAN逆变换编码器将单视角图像投影到潜在代码中，作为3D GAN的输入。然后，利用3D GAN生成的深度图对新视角进行变形。最后，开发一种新的SVINet，利用对称先验和同一潜在代码的多视角图像对应关系，对变形图像中的遮挡区域进行修复。定量和定性实验表明，我们的方法持续优于多种最先进的方法。

英文摘要：

3D GAN inversion projects a single image into the latent space of a pre-trained 3D GAN to achieve single-shot novel view synthesis, which requires visible regions with high fidelity and occluded regions with realism and multi-view consistency. However, existing methods focus on the reconstruction of visible regions, while the generation of occluded regions relies only on the generative prior of 3D GAN. As a result, the generated occluded regions often exhibit poor quality due to the information loss caused by the low bit-rate latent code. To address this, we introduce the warping-and-inpainting strategy to incorporate image inpainting into 3D GAN inversion and propose a novel 3D GAN inversion method, WarpGAN. Specifically, we first employ a 3D GAN inversion encoder to project the single-view image into a latent code that serves as the input to 3D GAN. Then, we perform warping to a novel view using the depth map generated by 3D GAN. Finally, we develop a novel SVINet, which leverages the symmetry prior and multi-view image correspondence w.r.t. the same latent code to perform inpainting of occluded regions in the warped image. Quantitative and qualitative experiments demonstrate that our method consistently outperforms several state-of-the-art methods.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。