CTRL-D：基于个性化2D扩散的可控动态3D场景编辑
CTRL-D: Controllable Dynamic 3D Scene Editing with Personalized 2D Diffusion
AI总结：
针对动态3D场景编辑不一致、可控性不足的问题，提出微调InstructPix2Pix并结合可变形3D Gaussians两阶段优化的框架，从单张参考图学习局部编辑，实现高质量时序一致编辑。
摘要：
Neural Radiance Fields和3D Gaussian Splatting等3D表示的最新进展，极大提升了真实感场景建模与新视角合成。然而，在动态3D场景中实现可控且一致的编辑仍是一项重大挑战。以往工作在很大程度上受限于其编辑骨干网络，导致编辑不一致且可控性有限。在我们的工作中，提出了一个新颖框架：首先微调InstructPix2Pix模型，随后基于可变形3D Gaussians对场景进行两阶段优化。该微调使模型能够从单张已编辑参考图像中“学习”编辑能力，将复杂的动态场景编辑任务转化为简单的2D图像编辑过程。通过直接从参考图像学习编辑区域与风格，我们的方法无需跟踪目标编辑区域即可实现一致、精确的局部编辑，有效解决动态场景编辑中的关键挑战。随后，两阶段优化逐步编辑已训练的动态场景，并使用专门设计的已编辑图像缓冲区加速收敛、提升时间一致性。与最先进方法相比，本方法提供更灵活、可控的局部场景编辑，取得高质量且一致的结果。
英文摘要：
Recent advances in 3D representations, such as Neural Radiance Fields and 3D Gaussian Splatting, have greatly improved realistic scene modeling and novel-view synthesis. However, achieving controllable and consistent editing in dynamic 3D scenes remains a significant challenge. Previous work is largely constrained by its editing backbones, resulting in inconsistent edits and limited controllability. In our work, we introduce a novel framework that first fine-tunes the InstructPix2Pix model, followed by a two-stage optimization of the scene based on deformable 3D Gaussians. Our fine-tuning enables the model to "learn" the editing ability from a single edited reference image, transforming the complex task of dynamic scene editing into a simple 2D image editing process. By directly learning editing regions and styles from the reference, our approach enables consistent and precise local edits without the need for tracking desired editing regions, effectively addressing key challenges in dynamic scene editing. Then, our two-stage optimization progressively edits the trained dynamic scene, using a designed edited image buffer to accelerate convergence and improve temporal consistency. Compared to state-of-the-art methods, our approach offers more flexible and controllable local scene editing, achieving high-quality and consistent results.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
