MeshPriorDiT：面向动作条件布料动力学的分层建模

中文摘要 AI 辅助

动作条件布料动力学预测既需要局部合理的变形，也需要长程协调。现有方法主要遵循两种范式：基于网格的图神经网络（GNN）通过材料连通性捕获局部物理响应，但其有限的消息传递范围限制了拓扑遥远区域间的协调，而自回归滚动预测易累积误差；基于Transformer的动力学模型通过全局注意力捕获长程交互，但通常无显式材料连通性，需直接从数据中学习局部拓扑响应。本文提出MeshPriorDiT，一种分层动力学模型，将未来布料运动分解为结构化网格先验与生成残差。动作条件网格GNN首先预测多步顶点位移，生成符合材料拓扑与抓取约束的参考轨迹；以历史状态、规划动作和网格先验为条件，残差DiT通过条件流匹配联合生成先验未捕获的残差运动，生成的残差进一步通过材料邻接关系缩放和解码，以协调相邻顶点的修正。我们在三个布料操作任务的15步自回归滚动预测上评估MeshPriorDiT，在三个任务的平均结果中，MeshPriorDiT相比仅GNN基线降低了43.42%的平均全局均方误差（Global MSE），相比DiT-DDPM基线降低了75.03%，同时保持了与仅GNN基线相当的有利边缘应变均方误差（Edge-strain MSE）。

英文摘要

Action-conditioned cloth dynamics prediction requires both locally plausible deformation and long-range coordination. Existing approaches largely follow two paradigms. Mesh-based GNNs capture local physical responses through material connectivity. However, their finite message-passing range limits coordination between topologically distant regions, while autoregressive rollouts tend to accumulate prediction errors. Transformer-based dynamics models capture long-range interactions through global attention, but often operate without explicit material connectivity and must learn local topological responses directly from data. We propose MeshPriorDiT, a hierarchical dynamics model that decomposes future cloth motion into a structured mesh prior and a generative residual. An action-conditioned mesh GNN first predicts multi-step vertex displacements, yielding a reference trajectory that respects material topology and grasp constraints. Conditioned on historical states, planned actions, and the mesh prior, a Residual DiT then uses conditional flow matching to jointly generate the residual motion not captured by the prior. The generated residual is further rescaled and decoded using material adjacency to coordinate corrections across neighboring vertices. We evaluate MeshPriorDiT on 15-step autoregressive rollouts across three cloth manipulation tasks. Averaged over the three tasks, MeshPriorDiT reduces average Global MSE by 43.42% relative to the GNN-Only baseline and by 75.03% relative to the DiT-DDPM baseline, while maintaining a favorable Edge-strain MSE comparable to that of GNN-Only.

发表机构

• Tsinghua University（清华大学）
• Dense-AI
• University of Michigan（密歇根大学）

机构由 AI 辅助整理，请以论文原文为准。