VGGT-Edit：基于残差场预测的前馈原生3D场景编辑
VGGT-Edit: Feed-forward Native 3D Scene Editing with Residual Field Prediction
AI总结：
本文提出VGGT-Edit，一种基于文本条件的前馈原生3D场景编辑框架，通过引入深度同步文本注入和残差变换头，实现高质量的3D场景编辑，同时构建DeltaScene数据集以提升编辑效果和推理速度。
摘要：
高质量的3D场景重建近年来已发展为通用的前馈架构，使单次正向传递即可生成复杂的环境。然而，尽管这些模型在静态场景感知方面表现强劲，但它们在响应动态人类指令方面仍然有限，限制了其在交互应用中的使用。现有的编辑方法通常依赖于2D提升策略，即单独编辑每个视图，然后将其提升回3D空间。这种间接流程往往导致模糊的纹理和不一致的几何结构，因为2D编辑器缺乏保持跨视角结构的空间意识。为了解决这些限制，我们提出了VGGT-Edit，一种用于文本条件的前馈框架，用于原生3D场景编辑。VGGT-Edit引入了深度同步的文本注入，以对齐语义指导与骨干网络的空间姿态，确保稳定的指令接地。此语义信号随后由残差变换头处理，直接预测3D几何位移以变形场景，同时保持背景稳定性。为了确保高保真结果，我们通过多术语目标函数监督该框架，强制几何准确性和跨视图一致性。我们还构建了DeltaScene数据集，一个通过自动化流程生成的大规模数据集，通过3D一致过滤确保地面真实质量。实验表明，VGGT-Edit在2D提升基线中表现显著更好，生成更清晰的物体细节，更强的多视图一致性以及接近即时的推理速度。项目页面是https://chriszkxxx.github.io/VGGT-Edit/.
英文摘要：
High-quality 3D scene reconstruction has recently advanced toward generalizable feed-forward architectures, enabling the generation of complex environments in a single forward pass. However, despite their strong performance in static scene perception, these models remain limited in responding to dynamic human instructions, which restricts their use in interactive applications. Existing editing methods typically rely on a 2D-lifting strategy, where individual views are edited independently and then lifted back into 3D space. This indirect pipeline often leads to blurry textures and inconsistent geometry, as 2D editors lack the spatial awareness required to preserve structure across viewpoints. To address these limitations, we propose VGGT-Edit, a feed-forward framework for text-conditioned native 3D scene editing. VGGT-Edit introduces depth-synchronized text injection to align semantic guidance with the backbone's spatial poses, ensuring stable instruction grounding. This semantic signal is then processed by a residual transformation head, which directly predicts 3D geometric displacements to deform the scene while preserving background stability. To ensure high-fidelity results, we supervise the framework with a multi-term objective function that enforces geometric accuracy and cross-view consistency. We also construct the DeltaScene Dataset, a large-scale dataset generated through an automated pipeline with 3D agreement filtering to ensure ground-truth quality. Experiments show that VGGT-Edit substantially outperforms 2D-lifting baselines, producing sharper object details, stronger multi-view consistency, and near-instant inference speed. The project page is https://chriszkxxx.github.io/VGGT-Edit/.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
