GS-VTON：基于高斯溅射的可控3D虚拟试穿
GS-VTON: Controllable 3D Virtual Try-on with Gaussian Splatting
AI总结：
针对3D虚拟试穿发展滞后、跨视角一致性差等问题，提出基于3D高斯溅射的图像提示3D VTON方法GS-VTON，含个性化扩散模型、人物感知3DGS编辑框架，还构建新基准3D-VTONBench，性能优于现有方法。
摘要：
基于扩散模型的2D虚拟试穿（VTON）技术近期展现出强劲性能，而3D VTON的发展则大幅滞后。尽管文本引导的3D场景编辑近期取得了进展，但将2D VTON整合到这些流程中以实现逼真的3D VTON仍具挑战性。原因有二：其一，文本提示无法提供足够的服装描述细节；其二，从同一场景不同视角生成的2D VTON结果缺乏连贯性与空间关联，因此常导致外观不一致和几何失真。为解决这些问题，我们提出一种图像提示的3D VTON方法（名为GS-VTON），该方法以3D高斯溅射（3D Gaussian Splatting，3DGS）作为3D表示，既能将2D VTON模型的预训练知识迁移至3D场景，又能提升跨视角一致性。(1) 具体而言，我们提出一种个性化扩散模型，利用低秩适配（LoRA）微调将个性化信息融入预训练2D VTON模型。为实现有效的LoRA训练，我们引入一种参考驱动的图像编辑方法，可在保证一致性的同时编辑多视角图像。(2) 此外，我们提出一个感知人物的3DGS编辑框架，在维持一致的跨视角外观与高质量3D几何结构的同时实现高效编辑。(3) 我们还构建了一个全新的3D VTON基准3D-VTONBench，用于开展全面的定性与定量3D VTON评估。通过大量实验及与现有方法的对比分析，所提方法展现出更优的保真度与先进的编辑能力，证实了其在3D VTON任务中的有效性。
英文摘要：
Diffusion-based 2D virtual try-on (VTON) techniques have recently demonstrated strong performance, while the development of 3D VTON has largely lagged behind. Despite recent advances in text-guided 3D scene editing, integrating 2D VTON into these pipelines to achieve vivid 3D VTON remains challenging. The reasons are twofold. First, text prompts cannot provide sufficient details in describing clothing. Second, 2D VTON results generated from different viewpoints of the same 3D scene lack coherence and spatial relationships, hence frequently leading to appearance inconsistencies and geometric distortions. To resolve these problems, we introduce an image-prompted 3D VTON method (dubbed GS-VTON) which, by leveraging 3D Gaussian Splatting (3DGS) as the 3D representation, enables the transfer of pre-trained knowledge from 2D VTON models to 3D while improving cross-view consistency. (1) Specifically, we propose a personalized diffusion model that utilizes low-rank adaptation (LoRA) fine-tuning to incorporate personalized information into pre-trained 2D VTON models. To achieve effective LoRA training, we introduce a reference-driven image editing approach that enables the simultaneous editing of multi-view images while ensuring consistency. (2) Furthermore, we propose a persona-aware 3DGS editing framework to facilitate effective editing while maintaining consistent cross-view appearance and high-quality 3D geometry. (3) Additionally, we have established a new 3D VTON benchmark, 3D-VTONBench, which facilitates comprehensive qualitative and quantitative 3D VTON evaluations. Through extensive experiments and comparative analyses with existing methods, the proposed \OM has demonstrated superior fidelity and advanced editing capabilities, affirming its effectiveness for 3D VTON.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
