arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2604.08042cs.CVcs.AI

3DrawAgent:通过早期对比经验教LLM进行3D绘制

3DrawAgent: Teaching LLM to Draw in 3D with Early Contrastive Experience

Beijing University of Posts and Telecommunications(北京邮电大学);Jiangnan University(江南大学);HaoHan Data(浩瀚数据)

Hongcan Xiao, Xinyue Xiao, Yilin Wang, Yue Zhang, Yonggang Qi

AI总结:

3DrawAgent通过几何反馈利用大语言模型生成3D贝塞尔曲线,引入相对经验优化策略,无需参数更新提升3D空间理解与绘制质量,实现免训练的3D草图智能发展。

摘要:

在三维空间中进行草图绘制能促进对形状、结构和空间关系的表达性推理,但通过自然语言生成3D草图仍是一个重大挑战。本文介绍了3DrawAgent,一种无需训练的语言驱动框架,利用大语言模型(LLMs)在几何反馈下依次绘制3D贝塞尔曲线。与以往的2D草图代理不同,我们的方法引入了相对经验优化策略,适应了最近提出的组奖励策略优化(GRPO)范式。不同于依赖显式地面真实监督,我们构建了生成草图之间的成对比较,每对由相对更好的和较差的结果组成,基于CLIP基于的感知奖励和LLM基于的细粒度定性评估。这些经验随后用于迭代优化3D绘制的先验知识,使模型能够通过黑盒强化提升其3D意识。这种设计使我们的模型能够在不更新参数的情况下自我改进其空间理解和绘制质量。实验表明,3DrawAgent能够从多样化的文本提示中生成复杂且连贯的3D贝塞尔草图,展现出涌现的几何推理能力,并能推广到新形状,建立了免训练 3D草图智能发展的新范式。

英文摘要:

Sketching in 3D space enables expressive reasoning about shape, structure, and spatial relationships, yet generating 3D sketches through natural language remains a major challenge. In this work, we introduce 3DrawAgent, a training-free, language-driven framework for 3D sketch generation that leverages large language models (LLMs) to sequentially draw 3D Bezier curves under geometric feedback. Unlike prior 2D sketch agents, our method introduces a relative experience optimization strategy that adapts the recently proposed Group Reward Policy Optimization (GRPO) paradigm. Instead of relying on explicit ground-truth supervision, we construct pairwise comparisons among generated sketches, with each pair consisting of a relatively better and a worse result based on CLIP-based perceptual rewards and LLM-based fine-grained qualitative assessment. These experiences are then used to iteratively refine the prior knowledge of 3D drawing, enabling black-box reinforcement of the model's 3D awareness. This design allows our model to self-improve its spatial understanding and drawing quality without parameter updates. Experiments show that 3DrawAgent can generate complex and coherent 3D Bezier sketches from diverse textual prompts, exhibit emergent geometric reasoning, and generalize to novel shapes, establishing a new paradigm for advancing the field of training-free 3D sketch intelligence.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑