统一扩散变压器用于高质量文本感知图像恢复
Unified Diffusion Transformer for High-fidelity Text-Aware Image Restoration
AI总结：
UniT通过整合扩散变压器、视觉-语言模型和文本识别模块，实现高质量文本感知图像恢复，有效抑制文本幻觉并取得最佳性能。
摘要：
文本感知图像恢复（TAIR）旨在从包含退化文本内容的低质量输入中恢复高质量图像。尽管扩散模型为一般图像恢复提供了强大的生成先验，但在以文本为中心的任务中，由于缺乏显式的语言知识，它们常常会产生文本幻觉。为此，我们提出了UniT，一种统一的文本恢复框架，通过迭代的方式整合扩散变压器（DiT）、视觉-语言模型（VLM）和文本识别模块（TSM），以实现高质量的文本恢复。在UniT中，VLM从退化图像中提取文本内容，提供显式的文本指导。同时，TSM在扩散特征上进行训练，在每次去噪步骤中生成中间OCR预测，使VLM在去噪过程中迭代优化其指导。最后，DiT主干利用其强大的表示能力，利用这些提示来恢复细粒度的文本内容，同时有效抑制文本幻觉。在SA-Text和Real-Text基准测试中，实验表明，UniT能够忠实重建退化文本，显著减少幻觉，并在TAIR任务中实现最先进的端到端F1分数性能。
英文摘要：
Text-Aware Image Restoration (TAIR) aims to recover high-quality images from low-quality inputs containing degraded textual content. While diffusion models provide strong generative priors for general image restoration, they often produce text hallucinations in text-centric tasks due to the absence of explicit linguistic knowledge. To address this, we propose UniT, a unified text restoration framework that integrates a Diffusion Transformer (DiT), a Vision-Language Model (VLM), and a Text Spotting Module (TSM) in an iterative fashion for high-fidelity text restoration. In UniT, the VLM extracts textual content from degraded images to provide explicit textual guidance. Simultaneously, the TSM, trained on diffusion features, generates intermediate OCR predictions at each denoising step, enabling the VLM to iteratively refine its guidance during the denoising process. Finally, the DiT backbone, leveraging its strong representational power, exploit these cues to recover fine-grained textual content while effectively suppressing text hallucinations. Experiments on the SA-Text and Real-Text benchmarks demonstrate that UniT faithfully reconstructs degraded text, substantially reduces hallucinations, and achieves state-of-the-art end-to-end F1-score performance in TAIR task.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
