Qffusion：通过象限网格注意力学习实现可控人像视频编辑
Qffusion: Controllable Portrait Video Editing via Quadrant-Grid Attention Learning
AI总结：
该研究提出双帧引导框架 Qffusion，通过象限网格排列与自注意力联合建模外观和时序，并以 QGP 递归推理实现稳定、任意长度的可控人像视频编辑。
摘要：
本文提出 Qffusion，一个用于人像视频编辑的双帧引导框架。具体而言，我们遵循“为编辑而动画化”的设计原则，将 Qffusion 从两张静态参考图像训练为一个通用动画框架；在推理时，只需将修改后的起始帧和结束帧作为参考，即可方便地将其用于人像视频编辑。借助 Stable Diffusion 强大的生成能力，我们提出了一种用于潜变量重排的象限网格排列（Quadrant-grid Arrangement，QGA）方案，将两张参考图像的潜变量编码与四种面部条件的潜变量编码分别排列成四宫格形式。随后，我们融合这两种模态的特征，并使用自注意力同时进行外观与时序学习，在 QGA 下联合建模不同时刻的表示。我们的 Qffusion 无需额外网络或复杂训练阶段即可实现稳定的视频编辑，仅修改了 Stable Diffusion 的输入格式。此外，我们提出象限网格传播（Quadrant-grid Propagation，QGP）推理策略，通过递归处理参考帧与条件帧，在稳定生成任意长度视频方面具有独特优势。大量实验表明，Qffusion 在人像视频编辑上持续优于现有最先进技术。项目主页：https://qffusion.github.io/page/。
英文摘要：
This paper presents Qffusion, a dual-frame-guided framework for portrait video editing. Specifically, we consider a design principle of ``animation for editing'', and train Qffusion as a general animation framework from two still reference images while we can use it for portrait video editing easily by applying modified start and end frames as references during inference. Leveraging the powerful generative power of Stable Diffusion, we propose a Quadrant-grid Arrangement (QGA) scheme for latent re-arrangement, which arranges the latent codes of two reference images and that of four facial conditions into a four-grid fashion, separately. Then, we fuse features of these two modalities and use self-attention for both appearance and temporal learning, where representations at different times are jointly modeled under QGA. Our Qffusion can achieve stable video editing without additional networks or complex training stages, where only the input format of Stable Diffusion is modified. Further, we propose a Quadrant-grid Propagation (QGP) inference strategy, which enjoys a unique advantage on stable arbitrary-length video generation by processing reference and condition frames recursively. Through extensive experiments, Qffusion consistently outperforms state-of-the-art techniques on portrait video editing. Project page: https://qffusion.github.io/page/.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
