3D表示与文本嵌入对齐用于3D内容个性化
Align 3D Representation and Text Embedding for 3D Content Personalization
AI总结：
针对3D内容个性化需昂贵再训练的问题，提出Invert3D框架，通过建立3D表示与文本嵌入空间的对齐，利用相机条件逆机制实现自然语言驱动的高效3D个性化，无需再训练。
摘要：
最近NeRF和3DGS的进展显著提升了3D内容合成的效率和质量，但生成的3D内容的高效个性化仍是关键挑战。当前3D个性化方法主要依赖基于知识蒸馏的方法，需计算成本高昂的再训练过程。为应对这一挑战，我们提出Invert3D，一种便捷的3D内容个性化新框架。如今，CLIP等视觉语言模型通过对齐的视觉-文本嵌入空间实现直接图像个性化，但3D内容与2D图像的固有结构差异阻碍了这些技术的直接应用。我们的方法通过建立3D表示与文本嵌入空间的对齐来弥合这一差距，具体而言，我们开发了一种相机条件的3D到文本逆机制，将3D内容投影到与文本嵌入对齐的3D嵌入中。这种对齐使通过自然语言提示高效操纵和个性化3D内容成为可能，无需计算密集的再训练过程。大量实验表明，Invert3D实现了3D内容的有效个性化。我们的工作可在https://github.com/qsong2001/Invert3D获取。
英文摘要：
Recent advances in NeRF and 3DGS have significantly enhanced the efficiency and quality of 3D content synthesis. However, efficient personalization of generated 3D content remains a critical challenge. Current 3D personalization approaches predominantly rely on knowledge distillation-based methods, which require computationally expensive retraining procedures. To address this challenge, we propose \textbf{Invert3D}, a novel framework for convenient 3D content personalization. Nowadays, vision-language models such as CLIP enable direct image personalization through aligned vision-text embedding spaces. However, the inherent structural differences between 3D content and 2D images preclude direct application of these techniques to 3D personalization. Our approach bridges this gap by establishing alignment between 3D representations and text embedding spaces. Specifically, we develop a camera-conditioned 3D-to-text inverse mechanism that projects 3D contents into a 3D embedding aligned with text embeddings. This alignment enables efficient manipulation and personalization of 3D content through natural language prompts, eliminating the need for computationally retraining procedures. Extensive experiments demonstrate that Invert3D achieves effective personalization of 3D content. Our work is available at: https://github.com/qsong2001/Invert3D.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
