arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2603.09312cs.CV
收藏

IntroSVG: 通过反思生成器-批评者框架从渲染反馈中学习以实现文本到SVG生成

IntroSVG: Learning from Rendering Feedback for Text-to-SVG Generation via an Introspective Generator-Critic Framework

Fudan University(复旦大学);TeleAI;Northwestern Polytechnical University(西北工业大学);University of Science and Technology of China(中国科学技术大学)

Feiyu Wang, Jiayuan Yang, Zhiyuan Zhao, Da Zhang, Bingyu Li, Peng Liu, Junyu Gao

AI总结:

IntroSVG通过反思生成器-批评者框架,结合监督微调和直接偏好优化,实现文本到SVG生成的高质量输出。

摘要:

可缩放矢量图形(SVG)因其固有的可缩放性和可编辑性在数字设计中占据核心地位。尽管视觉语言模型(VLMs)显著推动了内容生成的发展,但现有的文本到SVG生成方法仍受到核心挑战的限制:自回归训练过程不纳入最终渲染图像的视觉感知,从根本上限制了生成质量。为了解决这一限制,我们提出了一个反思SVG生成框架(IntroSVG)。其核心是实例化一个统一的VLM,在闭环中运作,同时承担生成器和批评者的双重角色。具体而言,通过监督微调(SFT),模型学习起草SVG并提供其渲染输出的反馈;此外,我们系统地将早期阶段的失败转换为高质量的错误纠正训练数据,从而增强模型的鲁棒性。随后,我们利用高容量的教师VLM构建偏好数据集,并进一步通过直接偏好优化(DPO)对生成器的策略进行对齐。在推理过程中,优化后的生成器和批评者在迭代的“生成-审查-完善”循环中协同工作,从不完美的中间草稿开始,自主提高输出质量。实验结果表明,我们的方法在多个关键评估指标上实现了最先进的性能,生成具有更复杂结构、更强语义对齐性和更高可编辑性的SVG。这些结果证实了将显式视觉反馈纳入生成循环的有效性。

英文摘要:

Scalable Vector Graphics (SVG) are central to digital design due to their inherent scalability and editability. Despite significant advancements in content generation enabled by Visual Language Models (VLMs), existing text-to-SVG generation methods are limited by a core challenge: the autoregressive training process does not incorporate visual perception of the final rendered image, which fundamentally constrains generation quality. To address this limitation, we propose an Introspective SVG Generation Framework (IntroSVG). At its core, the framework instantiates a unified VLM that operates in a closed loop, assuming dual roles of both generator and critic. Specifically, through Supervised Fine-Tuning (SFT), the model learns to draft SVGs and to provide feedback on their rendered outputs; moreover, we systematically convert early-stage failures into high-quality error-correction training data, thereby enhancing model robustness. Subsequently, we leverage a high-capacity teacher VLM to construct a preference dataset and further align the generator's policy through Direct Preference Optimization (DPO). During inference, the optimized generator and critic operate collaboratively in an iterative "generate-review-refine" cycle, starting from imperfect intermediate drafts to autonomously improve output quality. Experimental results demonstrate that our method achieves state-of-the-art performance across several key evaluation metrics, generating SVGs with more complex structures, stronger semantic alignment, and greater editability. These results corroborate the effectiveness of incorporating explicit visual feedback into the generation loop.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑