arXiv 2601.005352026-01-02cs.CV

FreeText: 通过注意力定位和频谱字形注入实现免训练的文本渲染

FreeText: Training-Free Text Rendering in Diffusion Transformers via Attention Localization and Spectral Glyph Injection

Anhui Province Key Laboratory of Digital Security, University of Science and Technology of China（安徽省数字安全重点实验室，中国科学技术大学）

Ruiqiang Zhang, Hengyi Wang, Chang Liu, Guanjie Wang, Zehua Ma, Weiming Zhang

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

FreeText通过注意力定位和频谱字形注入技术，无需训练即可提升文本渲染的准确性和美学质量。

摘要：

大规模的文本到图像（T2I）扩散模型在开放领域合成方面表现出色，但在精确文本渲染方面仍存在困难，尤其是在多行布局、密集字体和长尾脚本（如中文）方面。以往的解决方案通常需要昂贵的重新训练或刚性的外部布局约束，这会降低美观性和限制灵活性。我们提出FreeText，一个无需训练的即插即用框架，通过利用扩散变换器（DiT）模型的内在机制来提高文本渲染。FreeText将问题分解为“在哪里书写”和“书写什么”。对于“在哪里书写”，我们通过读取端内图像到文本注意力的逐词空间属性来定位书写区域，使用类似汇点的标记作为稳定的空间锚点，并通过拓扑感知的细化来生成高置信度的掩码。对于“书写什么”，我们引入了频谱调制字形注入（SGMI），它通过频域带通调制注入一个与噪声对齐的字形先验，以加强字形结构并抑制语义泄漏（渲染概念而不是单词）。在Qwen-Image、FLUX.1-dev和SD3变体上，针对长文本基准、CVTG以及我们自己的CLT-Bench进行了广泛的实验，结果在文本可读性方面取得了一致的提升，同时在很大程度上保持了语义对齐和审美质量，且推理开销较小。

英文摘要：

Large-scale text-to-image (T2I) diffusion models excel at open-domain synthesis but still struggle with precise text rendering, especially for multi-line layouts, dense typography, and long-tailed scripts such as Chinese. Prior solutions typically require costly retraining or rigid external layout constraints, which can degrade aesthetics and limit flexibility. We propose \textbf{FreeText}, a training-free, plug-and-play framework that improves text rendering by exploiting intrinsic mechanisms of \emph{Diffusion Transformer (DiT)} models. \textbf{FreeText} decomposes the problem into \emph{where to write} and \emph{what to write}. For \emph{where to write}, we localize writing regions by reading token-wise spatial attribution from endogenous image-to-text attention, using sink-like tokens as stable spatial anchors and topology-aware refinement to produce high-confidence masks. For \emph{what to write}, we introduce Spectral-Modulated Glyph Injection (SGMI), which injects a noise-aligned glyph prior with frequency-domain band-pass modulation to strengthen glyph structure and suppress semantic leakage (rendering the concept instead of the word). Extensive experiments on Qwen-Image, FLUX.1-dev, and SD3 variants across longText-Benchmark, CVTG, and our CLT-Bench show consistent gains in text readability while largely preserving semantic alignment and aesthetic quality, with modest inference overhead.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

AI总结：

摘要：

英文摘要：

相关论文