MVCustom: 通过几何潜在渲染和完成实现多视图定制化扩散
MVCustom: Multi-View Customized Diffusion via Geometric Latent Rendering and Completion
AI总结：
MVCustom通过几何潜在渲染和完成技术，实现多视图定制化扩散，解决多视图一致性和定制化保真度的统一问题。
摘要：
多视图生成与相机姿态控制及基于提示的定制化都是实现可控生成模型的重要元素。然而，现有的多视图生成模型不支持具有几何一致性的定制化，而定制化模型缺乏显式的视角控制，使它们难以统一。受这些差距的启发，我们引入了一个新的任务，即多视图定制化，旨在同时实现多视图相机姿态控制和定制化。由于定制化的训练数据稀缺，现有的多视图生成模型，其本质上依赖于大规模数据集，难以泛化到多样化的提示。为了解决这个问题，我们提出了MVCustom，一种新的基于扩散的框架，专门设计以实现多视图一致性和定制化保真度。在训练阶段，MVCustom通过特征场表示学习主体的身份和几何结构，结合增强的密集时空注意力文本到视频扩散骨干，利用时间一致性实现多视图一致性。在推理阶段，我们引入了两种新的技术：深度感知的特征渲染显式地强制几何一致性，以及一致性感知的潜在完成确保定制化主体和周围背景的准确视角对齐。广泛的实验表明，MVCustom在多视图一致性、定制化保真度方面实现了最平衡且一致的竞争性表现，展示了多目标生成任务的有效解决方案。
英文摘要：
Multi-view generation with camera pose control and prompt-based customization are both essential elements for achieving controllable generative models. However, existing multi-view generation models do not support customization with geometric consistency, whereas customization models lack explicit viewpoint control, making them challenging to unify. Motivated by these gaps, we introduce a novel task, multi-view customization, which aims to jointly achieve multi-view camera pose control and customization. Due to the scarcity of training data in customization, existing multi-view generation models, which inherently rely on large-scale datasets, struggle to generalize to diverse prompts. To address this, we propose MVCustom, a novel diffusion-based framework explicitly designed to achieve both multi-view consistency and customization fidelity. In the training stage, MVCustom learns the subject's identity and geometry using a feature-field representation, incorporating the text-to-video diffusion backbone enhanced with dense spatio-temporal attention, which leverages temporal coherence for multi-view consistency. In the inference stage, we introduce two novel techniques: depth-aware feature rendering explicitly enforces geometric consistency, and consistent-aware latent completion ensures accurate perspective alignment of the customized subject and surrounding backgrounds. Extensive experiments demonstrate that MVCustom achieves the most balanced and consistent competitive performance across multi-view consistency, customization fidelity, demonstrating effective solution of multi-objective generation task.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
