分块提示:克服图像和视频超分辨率中的提示误导
Tiled Prompts: Overcoming Prompt Misguidance in Image and Video Super-Resolution
AI总结:
本文提出Tiled Prompts框架,通过为每个潜在块生成特定提示,解决传统全局提示在超分辨率中的误导问题,提升图像和视频的感知质量和真实性。
摘要:
本文提出Tiled Prompts框架,通过为每个潜在块生成特定提示,解决传统全局提示在超分辨率中的误导问题,提升图像和视频的感知质量和真实性。文本条件扩散模型通过使用提示作为语义先验,推动了图像和视频超分辨率的发展,而现代超分辨率管道通常依赖于潜在块的分块来扩展到高分辨率。在实践中,单个全局提示与潜在分块结合使用,常常导致提示误导。具体而言,粗略的全局提示通常会遗漏局部细节(遗漏错误)并提供局部无关的指导(错误指导),这导致在分块级别产生次标准结果。为了解决这一问题,我们提出了Tiled Prompts,这是一个统一的图像和视频超分辨率框架,为每个潜在分块生成特定提示,并在局部文本条件后验下进行超分辨率处理,以最小的开销解决提示误导问题。我们的实验表明,在高分辨率真实图像和视频上,分块提示在感知质量和真实性方面带来了稳定的提升,同时减少了可以在全局提示基线中发现的幻觉和分块级伪影。
英文摘要:
Text-conditioned diffusion models have advanced image and video super-resolution by using prompts as semantic priors, and modern super-resolution pipelines typically rely on latent tiling to scale to high resolutions. In practice, a single global caption is used with the latent tiling, often causing prompt misguidance. Specifically, a coarse global prompt often misses localized details (errors of omission) and provides locally irrelevant guidance (errors of commission) which leads to substandard results at the tile level. To solve this, we propose Tiled Prompts, a unified framework for image and video super-resolution that generates a tile-specific prompt for each latent tile and performs super-resolution under locally text-conditioned posteriors to resolve prompt misguidance with minimal overhead. Our experiments on high resolution real-world images and videos show that tiled prompts bring consistent gains in perceptual quality and fidelity, while reducing hallucinations and tile-level artifacts that can be found in global-prompt baselines. Project Page: https://bryanswkim.github.io/tiled-prompts/.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。