InstantViR: 基于蒸馏扩散先验的实时视频逆问题求解器
InstantViR: Real-Time Video Inverse Problem Solver with Distilled Diffusion Prior
AI总结：
InstantViR通过蒸馏扩散先验实现高效实时视频重建，以35 FPS速度在NVIDIA A100 GPU上达到与基线相当的重建质量。
摘要：
视频逆问题对于流媒体、远程存在和AR/VR至关重要，其中高感知质量必须与严格的延迟约束共存。基于扩散的先验目前能够提供最先进的重建效果，但现有方法要么将图像扩散模型与随意的时序正则化器相结合，导致时序伪影，要么依赖于原生视频扩散模型，其迭代后验采样对于实时应用来说过于缓慢。我们介绍了InstantViR，一种用于超快视频重建的折衷推理框架，其动力来源于预训练的视频扩散先验。我们通过蒸馏一个强大的双向视频扩散模型（教师）到一个因果自回归学生，使降质视频能够直接在单次前向传递中映射到其恢复版本，继承教师强大的时序建模能力，同时完全去除迭代测试时间优化。蒸馏是先验驱动的：它只需要教师扩散模型和已知的退化操作符，不依赖外部配对的干净/噪声视频数据。为进一步提高吞吐量，我们通过创新的教师空间正则化蒸馏方案将视频扩散骨干VAE替换为高效LeanVAE，使低延迟的潜在空间处理成为可能。在流媒体随机修复、高斯去模糊和超分辨率等场景中，InstantViR在NVIDIA A100 GPU上以超过35 FPS的速度运行，比迭代视频扩散求解器快100倍，重建质量与基于扩散的基线相媲美。这些结果表明，基于扩散的视频重建可以与实时、交互式、可编辑、流媒体场景兼容，使高质量视频恢复成为现代视觉系统的一个实用组件。
英文摘要：
Video inverse problems are fundamental to streaming, telepresence, and AR/VR, where high perceptual quality must coexist with tight latency constraints. Diffusion-based priors currently deliver state-of-the-art reconstructions, but existing approaches either adapt image diffusion models with ad hoc temporal regularizers - leading to temporal artifacts - or rely on native video diffusion models whose iterative posterior sampling is far too slow for real-time use. We introduce InstantViR, an amortized inference framework for ultra-fast video reconstruction powered by a pre-trained video diffusion prior. We distill a powerful bidirectional video diffusion model (teacher) into a causal autoregressive student that maps a degraded video directly to its restored version in a single forward pass, inheriting the teacher's strong temporal modeling while completely removing iterative test-time optimization. The distillation is prior-driven: it only requires the teacher diffusion model and known degradation operators, and does not rely on externally paired clean/noisy video data. To further boost throughput, we replace the video-diffusion backbone VAE with a high-efficiency LeanVAE via an innovative teacher-space regularized distillation scheme, enabling low-latency latent-space processing. Across streaming random inpainting, Gaussian deblurring and super-resolution, InstantViR matches or surpasses the reconstruction quality of diffusion-based baselines while running at over 35 FPS on NVIDIA A100 GPUs, achieving up to 100 times speedups over iterative video diffusion solvers. These results show that diffusion-based video reconstruction is compatible with real-time, interactive, editable, streaming scenarios, turning high-quality video restoration into a practical component of modern vision systems.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
