SteerVTE: 具有风格和字形控制的无缝视频文本编辑
SteerVTE: Seamless Video Text Editing with Style and Glyph Control
AI总结：
提出SteerVTE框架，通过风格编码器和双粒度字形编码器控制冻结视频扩散模型，结合字形感知空间焦点损失和三阶段渐进训练，实现精确、连贯的视频文本编辑。
摘要：
视觉文本编辑旨在精确修改图像和视频中的文本，同时保持风格一致性和视觉真实性。尽管在图像领域取得了显著进展，视频文本编辑仍然很大程度上未被探索：这是一个局部任务，需要在小的文本区域内达到笔画级别的精度，这加剧了跨帧准确性、时间连贯性和风格保真度的挑战。我们提出了SteerVTE，一个统一的框架，通过风格和字形控制来引导冻结的视频扩散模型执行精确的视频文本编辑。基于冻结的扩散变换器，SteerVTE附加了一个轻量级的文本上下文适配器，包含两个互补模块：一个捕捉原始文本视觉属性的风格编码器，以及在线级和字符级编码目标文本的双粒度字形编码器。为了克服视频基础模型固有的弱文本渲染先验，我们进一步提出了一个字形感知的空间焦点损失和一个从图像到视频数据扩展的三阶段渐进训练课程。为了支持大规模训练，我们还开发了一个自动合成流程，并构建了SteerVTE-1M数据集，包含跨越不同场景、字体和风格效果的一百万个三元组。大量实验表明，SteerVTE在文本准确性、风格一致性和时间连贯性方面显著优于现有的视频编辑基线。
英文摘要：
Visual text editing aims to precisely modify text in images and videos while preserving stylistic consistency and visual realism. Despite significant advances in the image domain, video text editing remains largely unexplored: it is a localized task demanding stroke-level precision within small text regions, which compounds the challenges of cross-frame accuracy, temporal coherence, and stylistic fidelity. We introduce SteerVTE, a unified framework that \underline{\textbf{steer}}s a frozen video diffusion model to perform precise \underline{\textbf{V}}ideo \underline{\textbf{T}}ext \underline{\textbf{E}}diting through style and glyph control. Built on a frozen diffusion transformer, SteerVTE attaches a lightweight text context adapter with two complementary modules: a style encoder capturing the original text's visual attributes, and dual-granularity glyph encoders encoding the target text at both the line and character levels. To overcome the inherently weak text rendering priors of video foundation models, we further propose a glyph-aware spatial-focal loss and a three-stage progressive training curriculum that scales from image to video data. To support large-scale training, we also develop an automatic synthesis pipeline and construct SteerVTE-1M, a dataset of one million triplets spanning diverse scenes, fonts, and stylistic effects. Extensive experiments demonstrate that SteerVTE substantially outperforms existing video editing baselines across text accuracy, style consistency, and temporal coherence.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
