FLUX-Text: 一种简单且先进的扩散变换器基线用于场景文本编辑
FLUX-Text: A Simple and Advanced Diffusion Transformer Baseline for Scene Text Editing
AI总结：
FLUX-Text通过轻量级模块和两阶段训练策略，实现了高效多语言场景文本编辑，提升视觉质量和文本保真度。
摘要：
场景文本编辑旨在在图像上修改或添加文本，同时确保文本保真度和整体视觉质量与背景一致。最近的方法主要基于UNet-based扩散模型，这些模型已经提高了场景文本编辑的结果，但仍难以处理复杂的字符结构，尤其是非拉丁文字符（例如中文、韩文、日文）。为了解决这些问题，我们提出了FLUX-Text，一种简单且先进的多语言场景文本编辑DiT方法。具体来说，我们的FLUX-Text通过轻量级的视觉和文本嵌入模块增强了字符理解和生成，同时保持FLUX的原始生成能力。我们进一步提出了一种针对文本区域的区域文本感知损失，以及一种匹配的两阶段训练策略，以更好地平衡文本编辑和整体图像质量。得益于DiT架构和轻量级特征注入模块，FLUX-Text仅需0.1M训练示例即可训练，比流行方法所需的2.9M减少了97%。在多个公共数据集上的广泛实验，包括英语和中文基准，证明了我们的方法在视觉质量和文本保真度方面优于其他方法。所有代码均可在https://github.com/AMAP-ML/FluxText获取。
英文摘要：
Scene text editing aims to modify or add texts on images while ensuring text fidelity and overall visual quality consistent with the background. Recent methods are primarily built on UNet-based diffusion models, which have improved scene text editing results, but still struggle with complex glyph structures, especially for non-Latin ones (\eg, Chinese, Korean, Japanese). To address these issues, we present \textbf{FLUX-Text}, a simple and advanced multilingual scene text editing DiT method. Specifically, our FLUX-Text enhances glyph understanding and generation through lightweight Visual and Text Embedding Modules, while preserving the original generative capability of FLUX. We further propose a Regional Text Perceptual Loss tailored for text regions, along with a matching two-stage training strategy to better balance text editing and overall image quality. Benefiting from the DiT-based architecture and lightweight feature injection modules, FLUX-Text can be trained with only $0.1$M training examples, a \textbf{97\%} reduction compared to $2.9$M required by popular methods. Extensive experiments on multiple public datasets, including English and Chinese benchmarks, demonstrate that our method surpasses other methods in visual quality and text fidelity. All the code is available at https://github.com/AMAP-ML/FluxText.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
