Multimodal Large Language Models (MLLMs) have shown promising capabilities in generating Scalable Vector Graphics (SVG) via direct code synthesis. However, existing paradigms typically adopt an open-loop "blind drawing" approach, where models generate symbolic code sequences without perceiving intermediate visual outcomes. This methodology severely underutilizes the powerful visual priors embedded in MLLMs vision encoders, treating SVG generation as a disjointed textual sequence modeling task rather than an integrated visuo-spatial one. Consequently, models struggle to reason about partial canvas states and implicit occlusion relationships, which are visually explicit but textually ambiguous. To bridge this gap, we propose Render-in-the-Loop, a novel generation paradigm that reformulates SVG synthesis as a step-wise, visual-context-aware process. By rendering intermediate code states into a cumulative canvas, the model explicitly observes the evolving visual context at each step, leveraging on-the-fly feedback to guide subsequent generation. However, we demonstrate that applying this visual loop naively to off-the-shelf models is suboptimal due to their inability to leverage incremental visual-code mappings. To address this, we first utilize fine-grained path decomposition to construct dense multi-step visual trajectories, and then introduce a Visual Self-Feedback (VSF) training strategy to condition the next primitive generation on intermediate visual states. Furthermore, a Render-and-Verify (RaV) inference mechanism is proposed to effectively filter degenerate and redundant primitives. Our framework, instantiated on a multimodal foundation model, outperforms strong open-weight baselines on the standard MMSVGBench. This result highlights the remarkable data efficiency and generalization capability of our Render-in-the-Loop paradigm for both Text-to-SVG and Image-to-SVG tasks.
De-rendering, Reasoning, and Repairing Charts with Vision-Language Models
Universidad Torcuato Di Tella(托克托迪特拉大学);Consejo Nacional de Investigaciones Científicas y Técnicas(国家科学与技术研究理事会);Facultad de Ciencias Exactas y Naturales Universidad de Buenos Aires(布宜诺斯艾利斯大学科学与自然学院)
Image GANs meet Differentiable Rendering for Inverse Graphics and Interpretable 3D Neural Rendering
NVIDIA(英伟达);University of Toronto(多伦多大学);Vector Institute(向量研究所);University of Waterloo(滑铁卢大学);Stanford University(斯坦福大学);MIT CSAIL(麻省理工学院计算机科学与人工智能实验室)
IGD: Instructional Graphic Design with Multimodal Layer Generation
University of Science and Technology of China(中国科学技术大学);YuanShi Technology(元世科技);Institute of Trustworthy Embodied AI, Fudan University(复旦大学可信具身人工智能研究院)
Microsoft Research Asia(微软亚洲研究院);University of Technology Sydney(悉尼科技大学);The Hong Kong University of Science and Technology(香港科技大学)
在本文中,我们考虑一种不同的数据格式的图像:矢量图形。与广泛用于图像识别的光栅图形相比,矢量图形可以放大或缩小到任何分辨率,而不会出现混叠或信息丢失,这是由于文档中图元的解析表示。此外,矢量图形能够提供有关低级元素如何组合在一起以形成高级形状或结构的额外结构信息。在现有的方法中,图形向量的这些优点还没有被充分利用。为了探索这种数据格式,我们针对基本的识别任务:对象定位和分类。我们提出了一种高效的无CNN流水线,它不将图形渲染为像素(即光栅化),并将矢量图形的文本文档作为输入,称为YOLaT(You Only Look at Text)。YOLaT构建多图模型来描述矢量图形中的结构和空间信息,并提出了一种双流图神经网络来检测图中的对象。我们的实验表明,通过直接操作矢量图形,YOLaT优于基于光栅图形的目标检测基线的平均精度和效率。