arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2603.27448cs.LGcs.AIcs.CE
收藏

GIFT: 通过几何反馈提升图像到CAD程序合成

GIFT: Bootstrapping Image-to-CAD Program Synthesis via Geometric Feedback

DeCoDE Lab, MIT(麻省理工学院DeCoDE实验室);Core AI, IBM(IBM核心人工智能);MIT-IBM Watson AI Lab(MIT-IBM沃森人工智能实验室)

Giorgio Giannone, Anna Clare Doris, Amin Heyrani Nobari, Kai Xu, Akash Srivastava, Faez Ahmed

AI总结:

本文提出GIFT框架,通过几何反馈生成高质量训练样本,提升CAD程序生成的鲁棒性,无需额外标注。

摘要:

生成可执行的CAD程序需要视觉几何与符号程序表示的对齐,而当前方法在设计复杂度增加时无法可靠学习。现有微调方法依赖有限监督数据集或昂贵的后期训练流程,导致系统脆弱,限制了生成CAD设计的进步。我们指出,瓶颈不在于模型或算法能力,而在于缺乏对齐视觉几何与程序语法的多样化训练示例。此限制尤为突出,因为收集多样化且验证过的工程数据集既昂贵又难以扩展。我们引入几何推理反馈微调(GIFT),一种数据增强框架,利用几何反馈将测试时计算转化为增强的高质量训练样本。GIFT结合两种机制:Soft-Rejection Sampling(GIFT-REJECT),保留多样化的高保真程序,超越精确的地面实况匹配;Failure-Driven Augmentation(GIFT-FAIL),将近似预测转化为合成训练示例,提高对挑战性几何的鲁棒性。通过将推理时搜索 amortize 到模型参数中,GIFT在捕获测试时扩展的益处的同时,将推理计算减少了80%。它在强监督基线上将平均IoU提高12%,在不需额外人工标注或专用架构的情况下,仍与更复杂的多模态系统竞争。

英文摘要:

Generating executable CAD programs from images requires alignment between visual geometry and symbolic program representations, a capability that current methods fail to learn reliably as design complexity increases. Existing fine-tuning approaches rely on either limited supervised datasets or expensive post-training pipelines, resulting in brittle systems that restrict progress in generative CAD design. We argue that the primary bottleneck lies not in model or algorithmic capacity, but in the scarcity of diverse training examples that align visual geometry with program syntax. This limitation is especially acute because the collection of diverse and verified engineering datasets is both expensive and difficult to scale, constraining the development of robust generative CAD models. We introduce Geometric Inference Feedback Tuning (GIFT), a data augmentation framework that leverages geometric feedback to turn test-time compute into a bootstrapped set of high-quality training samples. GIFT combines two mechanisms: Soft-Rejection Sampling (GIFT-REJECT), which retains diverse high-fidelity programs beyond exact ground-truth matches, and Failure-Driven Augmentation (GIFT-FAIL), which converts near-miss predictions into synthetic training examples that improve robustness on challenging geometries. By amortizing inference-time search into the model parameters, GIFT captures the benefits of test-time scaling while reducing inference compute by 80%. It improves mean IoU by 12% over a strong supervised baseline and remains competitive with more complex multimodal systems, without requiring additional human annotation or specialized architectures.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑