条件在全头3D GANs中至关重要
Condition Matters in Full-head 3D GANs
AI总结：
本文提出使用视角不变的语义特征作为条件输入，以解决全头3D GANs中视角偏差问题，提升生成质量和一致性。
摘要：
在全头3D GANs的稳定训练中，条件至关重要。没有任何条件信号，模型会遭受严重的模式崩溃，使其训练变得不切实际。然而，一系列先前的全头3D GANs通常选择视角作为条件输入，这导致在条件视角方向上学习到的3D全头空间存在偏差。这在生成3D头的条件视角和非条件视角之间生成质量和多样性显著差异中表现出来，导致不同头区域的全局不一致性。在本工作中，我们提出使用视角不变的语义特征作为条件输入，从而将3D头的生成能力与视角方向解耦。为了为每个训练图像构建一个视角不变的语义条件，我们创建了一个新的合成头图像数据集。我们利用FLUX.1 Kontext将现有的高质量正面人脸数据集扩展到更宽的视角范围内。从正面视角提取的图像片段特征然后被用作扩展图像中所有视角的共享语义条件，确保语义对齐的同时消除方向偏差。这还允许同一主体不同视角的监督被统一到一个共享的语义条件下，这加快了训练并增强了生成3D头的全局一致性。此外，由于GANs在生成器学会几个成功欺骗判别器的模式后，多样性改进往往会变慢，我们的语义条件鼓励生成器遵循真实的语义分布，从而促进持续学习和多样化生成。在全头合成和单视角GAN反向过程中进行了广泛的实验，证明我们的方法在保真度、多样性和通用性方面显著更高。
英文摘要：
Conditioning is crucial for stable training of full-head 3D GANs. Without any conditioning signal, the model suffers from severe mode collapse, making it impractical to training. However, a series of previous full-head 3D GANs conventionally choose the view angle as the conditioning input, which leads to a bias in the learned 3D full-head space along the conditional view direction. This is evident in the significant differences in generation quality and diversity between the conditional view and non-conditional views of the generated 3D heads, resulting in global incoherence across different head regions. In this work, we propose to use view-invariant semantic feature as the conditioning input, thereby decoupling the generative capability of 3D heads from the viewing direction. To construct a view-invariant semantic condition for each training image, we create a novel synthesized head image dataset. We leverage FLUX.1 Kontext to extend existing high-quality frontal face datasets to a wide range of view angles. The image clip feature extracted from the frontal view is then used as a shared semantic condition across all views in the extended images, ensuring semantic alignment while eliminating directional bias. This also allows supervision from different views of the same subject to be consolidated under a shared semantic condition, which accelerates training and enhances the global coherence of the generated 3D heads. Moreover, as GANs often experience slower improvements in diversity once the generator learns a few modes that successfully fool the discriminator, our semantic conditioning encourages the generator to follow the true semantic distribution, thereby promoting continuous learning and diverse generation. Extensive experiments on full-head synthesis and single-view GAN inversion demonstrate that our method achieves significantly higher fidelity, diversity, and generalizability.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
