arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2603.03072cs.AIcs.CLcs.CV
收藏

TikZilla: 通过高质量数据和强化学习扩展文本到TikZ

TikZilla: Scaling Text-to-TikZ with High-Quality Data and Reinforcement Learning

University of Technology Nuremberg (UTN)(图恩堡技术大学)

Christian Greisinger, Steffen Eger

AI总结:

本文提出TikZilla,通过高质量数据和强化学习提升文本到TikZ的生成质量,解决现有数据小且噪声大、模型生成错误的问题。

摘要:

大型语言模型(LLMs)越来越多地被用于协助科学家在各种工作流中。一个关键挑战是根据文本描述生成高质量的图表,通常以TikZ程序表示,可渲染为科学图像。先前的研究提出了多种数据集和建模方法。然而,现有的文本到TikZ数据集太小且噪声大,无法捕捉TikZ的复杂性,导致文本与渲染图像不匹配。此外,先前的方法仅依赖于监督微调(SFT),这没有让模型接触到图像的渲染语义,常导致循环、无关内容和错误的空间关系。为了解决这些问题,我们构建了DaTikZ-V4,比DaTikZ-V3大四倍以上且质量显著提高,补充了LLM生成的图像描述。使用此数据集,我们训练了TikZilla,一个小型开源Qwen模型(3B和8B)的家族,采用SFT后接强化学习(RL)的两阶段流程。对于RL,我们利用通过逆图形训练的图像编码器提供语义忠实的奖励信号。大量的人类评估显示,TikZilla在5分制上比其基础模型提升1.5-2分,超越GPT-4o 0.5分,与GPT-5在图像评估中持平,同时模型规模小得多。代码、数据和模型将被公开。

英文摘要:

Large language models (LLMs) are increasingly used to assist scientists across diverse workflows. A key challenge is generating high-quality figures from textual descriptions, often represented as TikZ programs that can be rendered as scientific images. Prior research has proposed a variety of datasets and modeling approaches for this task. However, existing datasets for Text-to-TikZ are too small and noisy to capture the complexity of TikZ, causing mismatches between text and rendered figures. Moreover, prior approaches rely solely on supervised fine-tuning (SFT), which does not expose the model to the rendered semantics of the figure, often resulting in errors such as looping, irrelevant content, and incorrect spatial relations. To address these issues, we construct DaTikZ-V4, a dataset more than four times larger and substantially higher in quality than DaTikZ-V3, enriched with LLM-generated figure descriptions. Using this dataset, we train TikZilla, a family of small open-source Qwen models (3B and 8B) with a two-stage pipeline of SFT followed by reinforcement learning (RL). For RL, we leverage an image encoder trained via inverse graphics to provide semantically faithful reward signals. Extensive human evaluations with over 1,000 judgments show that TikZilla improves by 1.5-2 points over its base models on a 5-point scale, surpasses GPT-4o by 0.5 points, and matches GPT-5 in the image-based evaluation, while operating at much smaller model sizes.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑