Dynamic-eDiTor: 基于文本的无训练4D场景编辑与多模态扩散变换器
Dynamic-eDiTor: Training-Free Text-Driven 4D Scene Editing with Multimodal Diffusion Transformer
AI总结：
Dynamic-eDiTor通过多模态扩散变换器和4DGS实现无训练文本驱动的4D场景编辑，提升多视图和时间一致性。
摘要：
近年来，4D表示的进步，如动态NeRF和4D高斯点划法（4DGS），已使动态4D场景重建成为可能。然而，由于在空间和时间上确保多视图和时间一致性方面的挑战，文本驱动的4D场景编辑仍处于探索阶段。现有研究依赖于2D扩散模型，单独编辑帧，常导致运动失真、几何漂移和编辑不完整。我们引入了Dynamic-eDiTor，一种无训练的文本驱动4D编辑框架，利用多模态扩散变换器（MM-DiT）和4DGS。该机制包括空间-时间子网格注意力（STGA）用于局部一致的跨视图和时间融合，以及上下文令牌传播（CTP）通过令牌继承和光流引导的令牌替换进行全局传播。这些组件共同使Dynamic-eDiTor能够无缝、一致地进行多视图视频编辑，无需额外训练，并直接优化预训练的源4DGS。在多视图视频数据集DyNeRF上的大量实验表明，我们的方法在编辑保真度和多视图及时间一致性方面优于现有方法。结果和代码的项目页面：https://di-lee.github.io/dynamic-eDiTor/
英文摘要：
Recent progress in 4D representations, such as Dynamic NeRF and 4D Gaussian Splatting (4DGS), has enabled dynamic 4D scene reconstruction. However, text-driven 4D scene editing remains under-explored due to the challenge of ensuring both multi-view and temporal consistency across space and time during editing. Existing studies rely on 2D diffusion models that edit frames independently, often causing motion distortion, geometric drift, and incomplete editing. We introduce Dynamic-eDiTor, a training-free text-driven 4D editing framework leveraging Multimodal Diffusion Transformer (MM-DiT) and 4DGS. This mechanism consists of Spatio-Temporal Sub-Grid Attention (STGA) for locally consistent cross-view and temporal fusion, and Context Token Propagation (CTP) for global propagation via token inheritance and optical-flow-guided token replacement. Together, these components allow Dynamic-eDiTor to perform seamless, globally consistent multi-view video without additional training and directly optimize pre-trained source 4DGS. Extensive experiments on multi-view video dataset DyNeRF demonstrate that our method achieves superior editing fidelity and both multi-view and temporal consistency prior approaches. Project page for results and code: https://di-lee.github.io/dynamic-eDiTor/
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
