现实的画布，语言的画笔：从单目视频创建3D虚拟化身
Reality's Canvas, Language's Brush: Crafting 3D Avatars from Monocular Video
AI总结：
本文提出完全可微分管线ReCaLaB，仅从单一RGB视频学习高保真3D人类虚拟化身，结合可变形NeRF与文本提示控制位姿、纹理等，并利用扩散模型生成未见运动视频，在图像质量上超越以往单目方法。
摘要：
3D虚拟化身生成的最新进展在多视角监督下表现出色，能生成照片级真实感模型。然而，尽管单目方法适用性更广，其质量却相对落后。我们提出ReCaLaB来缩小这一差距。ReCaLaB是一个完全可微分的管线，仅从单一RGB视频即可学习高保真3D人类虚拟化身。该方法优化一个位姿条件的可变形NeRF，以体积方式在标准T-pose中表示人体对象。随后利用该标准表示，通过2D-3D对应关系高效关联神经纹理。这使得扩散颜色生成和光照校正分支得以分离，二者共同组合成RGB预测。该设计允许通过文本提示控制人体位姿、体型、纹理和光照等中间结果。图像条件的扩散模型由此帮助驱动3D虚拟化身的外观和位姿，以创建包含前所未见人体运动的视频序列。大量实验表明，在图像合成任务的图像质量方面，ReCaLaB优于以往的单目方法。此外，自然语言为3D人类虚拟化身的创意操控提供了直观的用户界面。
英文摘要：
Recent advancements in 3D avatar generation excel with multi-view supervision for photorealistic models. However, monocular counterparts lag in quality despite broader applicability. We propose ReCaLaB to close this gap. ReCaLaB is a fully-differentiable pipeline that learns high-fidelity 3D human avatars from just a single RGB video. A pose-conditioned deformable NeRF is optimized to volumetrically represent a human subject in canonical T-pose. The canonical representation is then leveraged to efficiently associate neural textures using 2D-3D correspondences. This enables the separation of diffused color generation and lighting correction branches that jointly compose an RGB prediction. The design allows to control intermediate results for human pose, body shape, texture, and lighting with text prompts. An image-conditioned diffusion model thereby helps to animate appearance and pose of the 3D avatar to create video sequences with previously unseen human motion. Extensive experiments show that ReCaLaB outperforms previous monocular approaches in terms of image quality for image synthesis tasks. Moreover, natural language offers an intuitive user interface for creative manipulation of 3D human avatars.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
