GaVS：通过时间一致的局部重建与渲染实现基于3D的视频稳像
GaVS: 3D-Grounded Video Stabilization via Temporally-Consistent Local Reconstruction and Rendering
AI总结：
GaVS提出基于3D的视频稳像方法，将稳像重构为时间一致的局部重建与渲染，利用高斯泼溅、测试时微调和场景外推避免裁剪，在几何一致性和用户体验上优于现有2D/2.5D方法。
摘要：
视频稳像在视频处理中至关重要，因为它能在保留用户原始运动意图的同时消除不必要的抖动。现有方法因其作用域不同而存在多种问题（如几何失真、过度裁剪、泛化能力差），从而降低用户体验。为解决这些问题，我们提出GaVS，一种新颖的基于3D的方法，将视频稳像重新表述为时间一致的“局部重建与渲染”范式。给定3D相机位姿，我们增强一个重建模型以预测高斯泼溅（Gaussian Splatting）基元，并在测试时通过多视角动态感知的光度监督和跨帧正则化对其进行微调，以生成时间一致的局部重建。随后使用该模型渲染每个稳像后的帧。我们利用场景外推模块来避免帧裁剪。我们的方法在一个重新构建的数据集上进行评估，该数据集注入了基于3D的信息，涵盖具有多样相机运动和场景动态的样本。定量上，在传统任务指标和新的几何一致性方面，我们的方法与最先进的2D和2.5D方法相比具有竞争力或更优。定性上，与替代方法相比，我们的方法产生了明显更好的结果，并通过用户研究得到验证。
英文摘要：
Video stabilization is pivotal for video processing, as it removes unwanted shakiness while preserving the original user motion intent. Existing approaches, depending on the domain they operate, suffer from several issues (e.g. geometric distortions, excessive cropping, poor generalization) that degrade the user experience. To address these issues, we introduce \textbf{GaVS}, a novel 3D-grounded approach that reformulates video stabilization as a temporally-consistent `local reconstruction and rendering' paradigm. Given 3D camera poses, we augment a reconstruction model to predict Gaussian Splatting primitives, and finetune it at test-time, with multi-view dynamics-aware photometric supervision and cross-frame regularization, to produce temporally-consistent local reconstructions. The model are then used to render each stabilized frame. We utilize a scene extrapolation module to avoid frame cropping. Our method is evaluated on a repurposed dataset, instilled with 3D-grounded information, covering samples with diverse camera motions and scene dynamics. Quantitatively, our method is competitive with or superior to state-of-the-art 2D and 2.5D approaches in terms of conventional task metrics and new geometry consistency. Qualitatively, our method produces noticeably better results compared to alternatives, validated by the user study.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
