SVCNet：具有时间聚合的基于涂鸦的视频上色网络
SVCNet: Scribble-based Video Colorization Network with Temporal Aggregation
AI总结：
本文提出 SVCNet，通过双阶段精确上色与时间平滑、联合分割及超分辨率模块，实现基于用户涂鸦的高质量、时序一致视频上色。
摘要：
本文提出了一种具有时间聚合的基于涂鸦的视频上色网络，名为 SVCNet。它可以根据用户提供的不同彩色涂鸦为单色视频上色。该网络解决了基于涂鸦的视频上色领域的三个常见问题：上色鲜艳度、时间一致性和颜色溢出。为提高上色质量并增强时间一致性，我们在 SVCNet 中采用两个顺序子网络，分别用于精确上色和时间平滑。第一阶段包含一个金字塔特征编码器，用于将彩色涂鸦与灰度帧结合，以及一个语义特征编码器，用于提取语义。第二阶段通过聚合相邻已上色帧（作为短程连接）和首个已上色帧（作为长程连接）的信息，对第一阶段的输出进行微调。为缓解颜色溢出伪影，我们同时学习视频上色和分割。此外，我们将大多数操作设置在固定的小图像分辨率上，并在 SVCNet 末端使用超分辨率模块恢复原始尺寸。这使 SVCNet 能够在推理时适应不同的图像分辨率。最后，我们在 DAVIS 和 Videvo 基准上评估所提出的 SVCNet。实验结果表明，与其他知名视频上色方法相比，SVCNet 能够生成质量更高且时间一致性更强的视频。代码和模型可在 https://github.com/zhaoyuzhi/SVCNet 获取。
英文摘要：
In this paper, we propose a scribble-based video colorization network with temporal aggregation called SVCNet. It can colorize monochrome videos based on different user-given color scribbles. It addresses three common issues in the scribble-based video colorization area: colorization vividness, temporal consistency, and color bleeding. To improve the colorization quality and strengthen the temporal consistency, we adopt two sequential sub-networks in SVCNet for precise colorization and temporal smoothing, respectively. The first stage includes a pyramid feature encoder to incorporate color scribbles with a grayscale frame, and a semantic feature encoder to extract semantics. The second stage finetunes the output from the first stage by aggregating the information of neighboring colorized frames (as short-range connections) and the first colorized frame (as a long-range connection). To alleviate the color bleeding artifacts, we learn video colorization and segmentation simultaneously. Furthermore, we set the majority of operations on a fixed small image resolution and use a Super-resolution Module at the tail of SVCNet to recover original sizes. It allows the SVCNet to fit different image resolutions at the inference. Finally, we evaluate the proposed SVCNet on DAVIS and Videvo benchmarks. The experimental results demonstrate that SVCNet produces both higher-quality and more temporally consistent videos than other well-known video colorization approaches. The codes and models can be found at https://github.com/zhaoyuzhi/SVCNet.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
