arXiv 2601.210812026-06-19cs.CV

思维形状：通过视觉思维链进行渐进式物体组装

Shape of Thought: Progressive Object Assembly via Visual Chain-of-Thought

School of Science and Engineering, The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳）科学与工程学院）；School of Data Science, The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳）数据科学学院）；Sun Yat-sen University（中山大学）；The Hong Kong University of Science and Technology, Guangzhou（香港科学与技术大学（广州））；Shenzhen Future Network of Intelligence Institute (FNii-Shenzhen)（深圳未来网络智能研究所（FNii-Shenzhen））；Guangdong Provincial Key Laboratory of Future Networks of Intelligence, CUHK(SZ)（广东省未来网络智能重点实验室，CUHK(SZ)）

Yu Huo, Siyu Zhang, Kun Zeng, Haoyue Liu, Owen Lee, Junlin Chen, Yuquan Lu, Yifu Guo, Yaodong Liang, Xiaoying Tang

备注：: ICML2026

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

提出Shape-of-Thought (SoT)框架，通过视觉思维链在渲染2D域中逐步组装形状，解决文本到图像生成中的组合结构约束问题，在组件计数和结构拓扑上显著优于直接生成。

摘要：

用于文本到图像生成的多模态模型已实现强视觉保真度，但在组合结构约束（特别是生成计数、属性绑定和部分级关系）下仍然脆弱。为解决这些挑战，我们提出了Shape-of-Thought (SoT)，一种视觉思维链框架，用于在渲染2D域中进行过程监督的渐进式形状组装，推理时无需外部引擎。SoT训练一个统一的多模态自回归模型，生成交错文本计划和渲染中间状态，帮助模型在不产生显式几何表示的情况下捕捉形状组装逻辑。与纯文本思维链不同，每个决策都基于渲染状态，使得计数、连接、拓扑和中间部件添加错误在整个轨迹中可检查。为支持这一范式，我们引入了SoT-26K，一个基于部件CAD层次结构的大规模接地组装轨迹数据集，以及T2S-CompBench，一个用于评估结构完整性和轨迹忠实度的基准。在SoT-26K上微调在组件计数上达到88.4%，在结构拓扑上达到84.8%，在组件计数上比直接生成高出24.2个百分点，在结构拓扑上高出19.3个百分点。SoT为渲染域结构感知生成建立了一个透明测试平台。代码见此https URL。

英文摘要：

Multimodal models for text-to-image generation have achieved strong visual fidelity, yet they remain brittle under compositional structural constraints, notably generative numeracy, attribute binding, and part-level relations. To address these challenges, we propose Shape-of-Thought (SoT), a visual CoT framework for process-supervised progressive shape assembly in the rendered 2D domain, without external engines at inference time. SoT trains a unified multimodal autoregressive model to generate interleaved textual plans and rendered intermediate states, helping the model capture shape-assembly logic without producing explicit geometric representations. Unlike text-only CoT, each decision is grounded in a rendered state, making counts, attachments, topology, and intermediate part-addition errors inspectable across the trajectory. To support this paradigm, we introduce SoT-26K, a large-scale dataset of grounded assembly traces derived from part-based CAD hierarchies, and T2S-CompBench, a benchmark for evaluating structural integrity and trace faithfulness. Fine-tuning on SoT-26K achieves 88.4% on component numeracy and 84.8% on structural topology, outperforming direct generation by +24.2 points on component numeracy and +19.3 points on structural topology. SoT establishes a transparent testbed for rendered-domain structure-aware generation. The code is available at https://github.com/yuhuo03/Shape-of-Thought.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。