通过将多视角重建网络缝合到视频生成器中实现文本到3D
Text-to-3D by Stitching a Multi-view Reconstruction Network to a Video Generator
AI总结：
VIST3A通过将多视角重建网络缝合到视频生成器中，实现文本到3D生成，提升3D场景生成质量。
摘要：
大规模预训练模型在视觉内容生成和3D重建领域的快速进展为文本到3D生成开辟了新可能。直观上，如果能够将现代潜在文本到视频模型作为
英文摘要：
The rapid progress of large, pretrained models for both visual content generation and 3D reconstruction opens up new possibilities for text-to-3D generation. Intuitively, one could obtain a formidable 3D scene generator if one were able to combine the power of a modern latent text-to-video model as "generator" with the geometric abilities of a recent (feedforward) 3D reconstruction system as "decoder". We introduce VIST3A, a general framework that does just that, addressing two main challenges. First, the two components must be joined in a way that preserves the rich knowledge encoded in their weights. We revisit model stitching, i.e., we identify the layer in the 3D decoder that best matches the latent representation produced by the text-to-video generator and stitch the two parts together. That operation requires only a small dataset and no labels. Second, the text-to-video generator must be aligned with the stitched 3D decoder, to ensure that the generated latents are decodable into consistent, perceptually convincing 3D scene geometry. To that end, we adapt direct reward finetuning, a popular technique for human preference alignment. We evaluate the proposed VIST3A approach with different video generators and 3D reconstruction models. All tested pairings markedly improve over prior text-to-3D models that output Gaussian splats. Moreover, by choosing a suitable 3D base model, VIST3A also enables high-quality text-to-pointmap generation.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
