向量化视频表示：通过层次化时空一致代理嵌入实现便捷编辑
Vectorized Video Representation with Easy Editing via Hierarchical Spatio-Temporally Consistent Proxy Embedding
AI总结：
本文提出一种基于层次化时空一致代理嵌入的向量化视频表示方法，以克服像素级跟踪的脆弱性，实现高精度重建与可控编辑。
摘要：
当前的视频表示方法在运动与外观建模上严重依赖不稳定且过度细粒度的先验，即像素级匹配与跟踪。仅几像素的跟踪误差就会导致视觉对象表示的崩溃，更不用说视频中频繁出现的遮挡和大范围运动。为克服上述脆弱性，本工作提出时空一致的代理节点来表示视频中动态变化的对象/场景。一方面，层次化代理节点能够稳定表达视觉对象的多尺度结构，因此不受累积跟踪误差、长期运动、遮挡和视角变化的影响。另一方面，代理节点的动态表示更新机制充分利用视频的时空先验来减轻不准确跟踪器的影响，从而有效处理场景和对象的剧烈变化。此外，对视频中不同视觉对象的形状和纹理表示采用解耦编码方式，有助于实现可控且细粒度的外观编辑能力。大量实验表明，所提出的表示以更少的参数实现了高视频重建精度，并支持复杂的视频处理任务，包括视频修复和基于关键帧的时域一致视频编辑。
英文摘要：
Current video representations heavily rely on unstable and over-grained priors for motion and appearance modelling, \emph{i.e.}, pixel-level matching and tracking. A tracking error of just a few pixels would lead to the collapse of the visual object representation, not to mention occlusions and large motion frequently occurring in videos. To overcome the above mentioned vulnerability, this work proposes spatio-temporally consistent proxy nodes to represent dynamically changing objects/scenes in the video. On the one hand, the hierarchical proxy nodes have the ability to stably express the multi-scale structure of visual objects, so they are not affected by accumulated tracking error, long-term motion, occlusion, and viewpoint variation. On the other hand, the dynamic representation update mechanism of the proxy nodes adequately leverages spatio-temporal priors of the video to mitigate the impact of inaccurate trackers, thereby effectively handling drastic changes in scenes and objects. Additionally, the decoupled encoding manner of the shape and texture representations across different visual objects in the video facilitates controllable and fine-grained appearance editing capability. Extensive experiments demonstrate that the proposed representation achieves high video reconstruction accuracy with fewer parameters and supports complex video processing tasks, including video in-painting and keyframe-based temporally consistent video editing.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
