In-N-Out：面向人脸编辑的基于体分解的可信3D GAN反演
In-N-Out: Faithful 3D GAN Inversion with Volumetric Decomposition for Face Editing
AI总结：
针对3D感知GAN反演中分布外人脸重建难、保真度与可编辑性难兼顾的问题，提出用双神经辐射场分别建模分布内外内容并优化合成的方法，提升了人脸重建与编辑效果。
摘要：
具备3D感知能力的GAN在保留其2D对应模型编辑功能的同时，为视图合成提供了新的能力。GAN反演是寻找潜在代码以重建输入图像或视频的关键步骤，随后可通过操纵该潜在代码实现多样的编辑任务。然而，在特定数据集（如FFHQ）上预训练的模型通常难以重建包含分布外（OOD）对象的图像，例如带有浓妆或遮挡物的人脸。我们通过在3D感知GAN中显式建模输入中的分布外对象来解决这一问题。我们的核心思路是使用两个独立的神经辐射场来表示图像：一个用于分布内内容，另一个用于分布外对象。最终的重建通过精心设计的正则化优化这两个辐射场的合成来实现。我们证明，这种显式分解缓解了重建保真度与可编辑性之间的固有权衡。我们在具有挑战性的真实人脸图像和视频上评估了所提方法的重建精度和可编辑性，并展示了其相较于其他基线方法的优异结果。
英文摘要：
3D-aware GANs offer new capabilities for view synthesis while preserving the editing functionalities of their 2D counterparts. GAN inversion is a crucial step that seeks the latent code to reconstruct input images or videos, subsequently enabling diverse editing tasks through manipulation of this latent code. However, a model pre-trained on a particular dataset (e.g., FFHQ) often has difficulty reconstructing images with out-of-distribution (OOD) objects such as faces with heavy make-up or occluding objects. We address this issue by explicitly modeling OOD objects from the input in 3D-aware GANs. Our core idea is to represent the image using two individual neural radiance fields: one for the in-distribution content and the other for the out-of-distribution object. The final reconstruction is achieved by optimizing the composition of these two radiance fields with carefully designed regularization. We demonstrate that our explicit decomposition alleviates the inherent trade-off between reconstruction fidelity and editability. We evaluate reconstruction accuracy and editability of our method on challenging real face images and videos and showcase favorable results against other baselines.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
