TextMastero:掌握多语言与多样式的高质量场景文本编辑
TextMastero: Mastering High-Quality Scene Text Editing in Diverse Languages and Styles
AI总结:
TextMastero基于潜在扩散模型,通过字形条件模块和潜在引导模块,解决多语言场景文本编辑中复杂字形生成不准确的问题,在保真度和风格相似性上超越现有方法。
摘要:
场景文本编辑旨在修改图像上的文本,同时保持新生成文本的风格与原始文本相似。给定一张图像、一个目标区域和目标文本,该任务会生成一张在选定区域中包含目标文本的输出图像,以替换原始文本。这一任务已被广泛研究,早期使用生成对抗网络(GANs)在文本保真度和风格相似性之间取得平衡,取得了初步成功。然而,基于GAN的方法在处理复杂背景或文本风格时存在困难。近期的工作利用扩散模型,展示了改进的结果,但仍面临挑战,尤其是在处理像CJK字符(中文、日文、韩文)这类具有复杂字形的非拉丁语言时,常常产生不准确或无法识别的字符。为了解决这些问题,我们提出了TextMastero——一种基于潜在扩散模型(LDMs)的精心设计的多语言场景文本编辑架构。TextMastero引入了两个关键模块:一个字形条件模块,用于在生成准确文本时进行细粒度的内容控制;以及一个潜在引导模块,用于提供全面的风格信息,以确保编辑前后的相似性。定性和定量实验均表明,我们的方法在文本保真度和风格相似性方面超越了所有已知的现有工作。
英文摘要:
Scene text editing aims to modify texts on images while maintaining the style of newly generated text similar to the original. Given an image, a target area, and target text, the task produces an output image with the target text in the selected area, replacing the original. This task has been studied extensively, with initial success using Generative Adversarial Networks (GANs) to balance text fidelity and style similarity. However, GAN-based methods struggled with complex backgrounds or text styles. Recent works leverage diffusion models, showing improved results, yet still face challenges, especially with non-Latin languages like CJK characters (Chinese, Japanese, Korean) that have complex glyphs, often producing inaccurate or unrecognizable characters. To address these issues, we present \emph{TextMastero} - a carefully designed multilingual scene text editing architecture based on latent diffusion models (LDMs). TextMastero introduces two key modules: a glyph conditioning module for fine-grained content control in generating accurate texts, and a latent guidance module for providing comprehensive style information to ensure similarity before and after editing. Both qualitative and quantitative experiments demonstrate that our method surpasses all known existing works in text fidelity and style similarity.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。