4DGS-Craft: 具备一致性与交互性的4D高斯泼溅编辑
4DGS-Craft: Consistent and Interactive 4D Gaussian Splatting Editing
AI总结：
针对4D高斯泼溅编辑存在的多维度一致性不足、复杂指令处理难的问题，提出4DGS-Craft框架，通过4D感知编辑模型、多视图优化、高斯选择机制及LLM意图解析，实现更一致可控的交互式4D场景编辑。
摘要：
当前4D高斯泼溅（4D Gaussian Splatting, 4DGS）编辑领域的最新进展仍面临视图一致性、时间一致性、非编辑区域一致性不足，以及复杂文本指令处理困难的挑战。为解决这些问题，我们提出4DGS-Craft——一个具备一致性与交互性的4DGS编辑框架。我们首先引入具备4D感知能力的InstructPix2Pix模型，以同时保障视图与时间一致性，该模型融合了从初始场景中提取的4D VGGT几何特征，使其能够在编辑过程中捕捉底层4D几何结构。我们进一步通过多视图网格模块增强该模型性能，该模块通过迭代优化多视图输入图像、同时联合优化底层4D场景来强化一致性。此外，我们通过一种新型高斯选择机制保障非编辑区域的一致性，该机制仅识别并优化编辑区域内的高斯体。除一致性外，提升用户交互便捷性对高效的4DGS编辑也至关重要。因此我们设计了基于大语言模型（LLM）的用户意图理解模块，该模块采用用户指令模板定义原子编辑操作，并利用LLM进行推理。最终我们的框架能够解析用户意图，将复杂指令分解为逻辑连贯的原子操作序列，从而处理复杂的用户指令并进一步提升编辑性能。与相关工作相比，我们的方法可实现一致性更强、可控性更高的4D场景编辑，代码将在论文录用后公开。
英文摘要：
Recent advances in 4D Gaussian Splatting (4DGS) editing still face challenges with view, temporal, and non-editing region consistency, as well as with handling complex text instructions. To address these issues, we propose 4DGS-Craft, a consistent and interactive 4DGS editing framework. We first introduce a 4D-aware InstructPix2Pix model to ensure both view and temporal consistency. This model incorporates 4D VGGT geometry features extracted from the initial scene, enabling it to capture underlying 4D geometric structures during editing. We further enhance this model with a multi-view grid module that enforces consistency by iteratively refining multi-view input images while jointly optimizing the underlying 4D scene. Furthermore, we preserve the consistency of non-edited regions through a novel Gaussian selection mechanism, which identifies and optimizes only the Gaussians within the edited regions. Beyond consistency, facilitating user interaction is also crucial for effective 4DGS editing. Therefore, we design an LLM-based module for user intent understanding. This module employs a user instruction template to define atomic editing operations and leverages an LLM for reasoning. As a result, our framework can interpret user intent and decompose complex instructions into a logical sequence of atomic operations, enabling it to handle intricate user commands and further enhance editing performance. Compared to related works, our approach enables more consistent and controllable 4D scene editing. Our code will be made available upon acceptance.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
