面向3D感知生成对抗网络的多视角反演
Multi-view Inversion for 3D-aware Generative Adversarial Networks
AI总结：
针对现有3D GAN人头反演仅用单张正面图、遗漏多视角信息的问题，提出多视角反演方法，通过多隐码扩展处理动态视频不一致性，提升几何精度与图像质量，且反演结果具备可编辑性。
摘要：
当前针对人头的3D GAN反演方法通常仅使用单张正面图像来重建完整的3D人头模型。当存在多视角数据或动态视频时，这种方式会遗漏有价值的信息。我们的方法基于现有的前沿3D GAN反演技术，能够对同一目标的多个视角进行一致的同步反演。我们采用多隐码扩展方案来处理动态人脸视频中存在的不一致性，从序列中重新合成一致的3D表示。由于我们的方法利用了目标主体的额外信息，我们观察到其在几何精度和图像质量上均有显著提升，尤其是在大视角渲染时。此外，我们验证了反演得到的3D渲染结果具备可编辑性，这是其与基于NeRF的场景重建的区别所在。
英文摘要：
Current 3D GAN inversion methods for human heads typically use only one single frontal image to reconstruct the whole 3D head model. This leaves out meaningful information when multi-view data or dynamic videos are available. Our method builds on existing state-of-the-art 3D GAN inversion techniques to allow for consistent and simultaneous inversion of multiple views of the same subject. We employ a multi-latent extension to handle inconsistencies present in dynamic face videos to re-synthesize consistent 3D representations from the sequence. As our method uses additional information about the target subject, we observe significant enhancements in both geometric accuracy and image quality, particularly when rendering from wide viewing angles. Moreover, we demonstrate the editability of our inverted 3D renderings, which distinguishes them from NeRF-based scene reconstructions.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
