GlyphMastero：一种用于高保真场景文本编辑的字形编码器
GlyphMastero: A Glyph Encoder for High-Fidelity Scene Text Editing
AI总结：
本文提出字形编码器GlyphMastero，通过字形注意力模块和特征金字塔网络捕捉字符跨层级与多尺度特征，引导扩散模型实现笔画级精度的场景文本编辑，在句子准确率和FID指标上显著超越现有基线。
摘要：
场景文本编辑是图像编辑的一个子领域，要求在修改图像中文本的同时保持风格一致性以及与周围环境的视觉连贯性。尽管基于扩散模型的方法在文本生成方面已展现出潜力，但它们仍然难以产生高质量的结果。这些方法常常生成扭曲或无法识别的字符，尤其是在处理像中文这样的复杂字符时。在此类系统中，字符由复杂的笔画模式和空间关系构成，必须被精确保持。我们提出GlyphMastero，一种专门设计的字形编码器，用于引导潜在扩散模型以笔画级精度生成文本。我们的核心见解是，现有方法尽管使用预训练的OCR模型进行特征提取，却未能捕捉文本结构的层次性——从单个笔画到笔画级交互，再到整体字符级结构。为解决这一问题，我们的字形编码器通过新颖的字形注意力模块，显式建模并捕捉局部级单个字符与全局级文本行之间的跨层级交互。同时，我们的模型实现了特征金字塔网络，在全局级融合多尺度OCR骨干特征。通过这些跨层级和多尺度融合，我们获得了更细致的字形感知引导，从而能够精确控制场景文本生成过程。我们的方法在句子准确率上比最先进的多语言场景文本编辑基线提高了18.02%，同时将文本区域的Fréchet inception distance降低了53.28%。
英文摘要：
Scene text editing, a subfield of image editing, requires modifying texts in images while preserving style consistency and visual coherence with the surrounding environment. While diffusion-based methods have shown promise in text generation, they still struggle to produce high-quality results. These methods often generate distorted or unrecognizable characters, particularly when dealing with complex characters like Chinese. In such systems, characters are composed of intricate stroke patterns and spatial relationships that must be precisely maintained. We present GlyphMastero, a specialized glyph encoder designed to guide the latent diffusion model for generating texts with stroke-level precision. Our key insight is that existing methods, despite using pretrained OCR models for feature extraction, fail to capture the hierarchical nature of text structures - from individual strokes to stroke-level interactions to overall character-level structure. To address this, our glyph encoder explicitly models and captures the cross-level interactions between local-level individual characters and global-level text lines through our novel glyph attention module. Meanwhile, our model implements a feature pyramid network to fuse the multi-scale OCR backbone features at the global-level. Through these cross-level and multi-scale fusions, we obtain more detailed glyph-aware guidance, enabling precise control over the scene text generation process. Our method achieves an 18.02\% improvement in sentence accuracy over the state-of-the-art multi-lingual scene text editing baseline, while simultaneously reducing the text-region Fréchet inception distance by 53.28\%.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
