超越对齐：通过解析引导的时间相干Transformer进行盲视频人脸恢复
Beyond Alignment: Blind Video Face Restoration via Parsing-Guided Temporal-Coherent Transformer
AI总结：
提出首个无需预对齐的盲视频人脸恢复方法PGTFormer，通过语义解析引导选择最优先验，结合时空向量量化自编码器和时间保真度调节器，生成时间相干且无伪影的高质量恢复结果，优于现有基线。
摘要：
现实世界中低质量视频人脸存在多种复杂退化的耦合。因此，盲视频人脸恢复是一个极具挑战性的病态问题，不仅需要幻觉生成高保真细节，还需在各种姿态变化中增强时间相干性。以简单方式独立恢复每一帧不可避免地会因姿态变化和关键点定位误差引入时间不相干和伪影。为此，我们提出首个无需预对齐的盲视频人脸恢复方法——一种新颖的解析引导时间相干Transformer（PGTFormer）。PGTFormer利用语义解析引导选择最优人脸先验，以生成时间相干且无伪影的结果。具体而言，我们在高质量视频人脸数据集上预训练一个时空向量量化自编码器，以提取富有表现力且上下文丰富的先验。随后，时间解析引导码本预测器（TPCP）基于人脸解析上下文线索恢复不同姿态下的人脸，而无需执行人脸预对齐。该策略减少了伪影，并缓解了由人脸预对齐累积误差引起的抖动。最后，时间保真度调节器（TFR）通过时间特征交互增强保真度，并提升视频时间一致性。在人脸视频上的大量实验表明，我们的方法优于以往的人脸恢复基线。代码将发布于https://github.com/kepengxu/PGTFormer。
英文摘要：
Multiple complex degradations are coupled in low-quality video faces in the real world. Therefore, blind video face restoration is a highly challenging ill-posed problem, requiring not only hallucinating high-fidelity details but also enhancing temporal coherence across diverse pose variations. Restoring each frame independently in a naive manner inevitably introduces temporal incoherence and artifacts from pose changes and keypoint localization errors. To address this, we propose the first blind video face restoration approach with a novel parsing-guided temporal-coherent transformer (PGTFormer) without pre-alignment. PGTFormer leverages semantic parsing guidance to select optimal face priors for generating temporally coherent artifact-free results. Specifically, we pre-train a temporal-spatial vector quantized auto-encoder on high-quality video face datasets to extract expressive context-rich priors. Then, the temporal parse-guided codebook predictor (TPCP) restores faces in different poses based on face parsing context cues without performing face pre-alignment. This strategy reduces artifacts and mitigates jitter caused by cumulative errors from face pre-alignment. Finally, the temporal fidelity regulator (TFR) enhances fidelity through temporal feature interaction and improves video temporal consistency. Extensive experiments on face videos show that our method outperforms previous face restoration baselines. The code will be released on \href{https://github.com/kepengxu/PGTFormer}{https://github.com/kepengxu/PGTFormer}.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
