LLV-FSR：利用大型语言-视觉先验进行人脸超分辨率
LLV-FSR: Exploiting Large Language-Vision Prior for Face Super-resolution
AI总结：
本文提出LLV-FSR框架，结合大型视觉-语言模型与高阶视觉先验生成多元化线索，以增强特征表示并显著提升人脸超分辨率的重建与感知质量。
摘要：
现有的人脸超分辨率（FSR）方法取得了显著进展，但它们主要利用有限的视觉信息（特别是原始像素空间）进行超分辨率，通常忽略了多元化线索，如高阶深度和语义，以及非视觉输入（文本标题和描述）。因此，这些方法难以从输入人脸中产生统一且有意义的表示。我们假设，将语言-视觉多元表示引入未开发的潜在嵌入空间，可以通过编码和利用语言-视觉先验间的互补性来增强 FSR。这促使我们提出了一个名为 LLV-FSR 的新框架，它将大型视觉-语言模型和高阶视觉先验的能力与具有挑战性的 FSR 任务相结合。具体而言，除了直接从原始输入中吸收知识外，我们还引入了预训练的视觉-语言模型来生成多元化先验，包括图像标题、描述、人脸语义掩码和深度。这些先验随后被用于指导更关键的特征表示，促进逼真且高质量的人脸超分辨率。实验结果表明，我们提出的框架显著提高了重建质量和感知质量，在 MMCelebA-HQ 数据集上的 PSNR 方面超越了 SOTA 0.43dB。
英文摘要：
Existing face super-resolution (FSR) methods have made significant advancements, but they primarily super-resolve face with limited visual information, original pixel-wise space in particular, commonly overlooking the pluralistic clues, like the higher-order depth and semantics, as well as non-visual inputs (text caption and description). Consequently, these methods struggle to produce a unified and meaningful representation from the input face. We suppose that introducing the language-vision pluralistic representation into unexplored potential embedding space could enhance FSR by encoding and exploiting the complementarity across language-vision prior. This motivates us to propose a new framework called LLV-FSR, which marries the power of large vision-language model and higher-order visual prior with the challenging task of FSR. Specifically, besides directly absorbing knowledge from original input, we introduce the pre-trained vision-language model to generate pluralistic priors, involving the image caption, descriptions, face semantic mask and depths. These priors are then employed to guide the more critical feature representation, facilitating realistic and high-quality face super-resolution. Experimental results demonstrate that our proposed framework significantly improves both the reconstruction quality and perceptual quality, surpassing the SOTA by 0.43dB in terms of PSNR on the MMCelebA-HQ dataset.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
