MoViE: 面向视频编辑的移动端扩散模型
MoViE: Mobile Diffusion for Video Editing
AI总结：
针对扩散视频编辑模型难以在移动端部署的问题，本研究通过架构优化、多模态无分类器引导蒸馏、对抗蒸馏减采样步等方案，实现移动端12帧/秒的高质量视频编辑。
摘要：
基于扩散的视频编辑技术近期取得的进展已展现出落地应用的巨大潜力，但这类方法计算成本极高，难以在移动设备上部署。本研究提出一系列优化方案，让移动端视频编辑成为可能。我们以现有图像编辑模型为基础，首先优化其架构并引入轻量自编码器；随后将无分类器引导蒸馏扩展至多模态，实现设备端推理速度三倍提升；最后通过一种新型对抗蒸馏方案将采样步数降至1，同时保留编辑过程的可控性。这些优化共同实现了移动设备上每秒12帧的视频编辑速度，且编辑质量保持较高水平。研究成果可查看：https://qualcomm-ai-research.github.io/mobile-video-editing/
英文摘要：
Recent progress in diffusion-based video editing has shown remarkable potential for practical applications. However, these methods remain prohibitively expensive and challenging to deploy on mobile devices. In this study, we introduce a series of optimizations that render mobile video editing feasible. Building upon the existing image editing model, we first optimize its architecture and incorporate a lightweight autoencoder. Subsequently, we extend classifier-free guidance distillation to multiple modalities, resulting in a threefold on-device speedup. Finally, we reduce the number of sampling steps to one by introducing a novel adversarial distillation scheme which preserves the controllability of the editing process. Collectively, these optimizations enable video editing at 12 frames per second on mobile devices, while maintaining high quality. Our results are available at https://qualcomm-ai-research.github.io/mobile-video-editing/
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
