PercHead：用于单图像3D头部重建与编辑的感知头部模型
PercHead: Perceptual Head Model for Single-Image 3D Head Reconstruction & Editing
AI总结：
提出PercHead方法，通过基于DINOv2和SAM2.1的感知监督策略及Gaussian Splatting渲染，实现高质量的单图像3D头部重建，并支持通过分割图和文本或图像提示进行直观的语义3D编辑。
摘要：
我们提出了PercHead，一种用于单图像3D头部重建和语义3D编辑的方法——由于严重的视角遮挡、微弱的感知监督以及3D空间编辑的模糊性，这两项任务本身就极具挑战性。我们开发了一个统一的基础模型，用于从单张输入图像重建视角一致的3D头部。该模型采用双分支编码器，随后接一个基于ViT的解码器，通过迭代交叉注意力将2D特征提升至3D空间。渲染使用Gaussian Splatting执行。我们方法的核心是一种基于DINOv2和SAM2.1的新型感知监督策略，它为几何和外观保真度提供了丰富且泛化的信号。我们的模型在新视角合成方面达到了最先进的性能，并且与现有基线相比，对极端视角展现出了卓越的鲁棒性。此外，通过替换编码器并微调网络，该基础模型可以无缝扩展用于语义3D编辑。在此变体中，我们通过两种不同的输入模态解耦几何和风格：使用分割图控制几何，并使用文本提示或参考图像指定外观。我们通过轻量级的交互式GUI突出了模型直观且强大的3D编辑能力，用户可以通过绘制分割图轻松雕刻几何，并通过自然语言或图像提示对外观进行风格化。
英文摘要：
We present PercHead, a method for single-image 3D head reconstruction and semantic 3D editing - two tasks that are inherently challenging due to severe view occlusions, weak perceptual supervision, and the ambiguity of editing in 3D space. We develop a unified base model for reconstructing view-consistent 3D heads from a single input image. The model employs a dual-branch encoder followed by a ViT-based decoder that lifts 2D features into 3D space through iterative cross-attention. Rendering is performed using Gaussian Splatting. At the heart of our approach is a novel perceptual supervision strategy based on DINOv2 and SAM2.1, which provides rich, generalized signals for both geometric and appearance fidelity. Our model achieves state-of-the-art performance in novel-view synthesis and, furthermore, exhibits exceptional robustness to extreme viewing angles compared to established baselines. Furthermore, this base model can be seamlessly extended for semantic 3D editing by swapping the encoder and finetuning the network. In this variant, we disentangle geometry and style through two distinct input modalities: a segmentation map to control geometry and either a text prompt or a reference image to specify appearance. We highlight the intuitive and powerful 3D editing capabilities of our model through a lightweight, interactive GUI, where users can effortlessly sculpt geometry by drawing segmentation maps and stylize appearance via natural language or image prompts. Project Page: https://antoniooroz.github.io/PercHead Video: https://www.youtube.com/watch?v=4hFybgTk4kE
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
