PINT：一种具有共享预测意图的机器人操作统一模型
PINT: A Unified Model with Shared Predictive Intent for Robot Manipulation
AI总结：
PINT是一种带共享预测意图的机器人操作统一模型，通过分块级预测意图协调VLM、VGM与AE，经两阶段训练后，在RoboTwin 2.0-Hard、LIBERO-Plus及真实双臂任务上的ID与OOD性能均优于对比模型。
摘要：
通用机器人操作可受益于用于语义推理的预训练视觉语言模型（VLM）和用于世界建模的视频生成模型（VGM）。然而，现有通过多层共享注意力直接融合两者的统一模型，更难利用它们的专业能力。我们提出PINT，一种通过共享分块级预测意图协调语义推理、世界建模和动作生成，同时保留专家特定通路的统一模型。PINT将可学习的意图令牌插入VLM序列以形成该表示，同时对VGM和动作专家（AE）进行条件设置；它仅替换VGM的提示条件，推理时保留VGM，AE通过单向注意力读取其表示。PINT不规定预测意图应编码的内容，而是从VLM、VGM和AE的原生预测目标中联合学习，这些目标提供了同一即将执行任务的互补视角。训练分为两个阶段：第一阶段冻结VGM，使未来视频预测通过任务相关的未来动态塑造预测意图；第二阶段固定学习到的意图，适配VGM和AE。在无具身预训练的情况下，PINT在分布内（ID）表现强劲，且大幅提升分布外（OOD）泛化性能：在RoboTwin 2.0-Hard上达到15.5%，而Fast-WAM为1.8%、StarVLA为3.2%；在LIBERO-Plus上达到79.5%，对应对比模型分别为51.5%和75.0%。在五项真实世界双臂任务中，PINT的ID成功率为68%、OOD成功率为24%，而预训练的Motus为35%/9%、预训练的π₀.₅为67%/35%。
英文摘要：
Generalist robot manipulation can benefit from pretrained vision-language models (VLMs) for semantic reasoning and video generation models (VGMs) for world modeling. However, existing unified models that directly fuse them through shared attention at multiple layers can make it harder to draw on their specialized capabilities. We introduce PINT, a unified model that coordinates semantic reasoning, world modeling, and action generation through a shared chunk-level Predictive Intent while preserving expert-specific pathways. PINT inserts learnable Intent Tokens into the VLM sequence to form this representation, conditioning both the VGM and an Action Expert (AE). It replaces only the VGM's prompt conditioning and retains the VGM at inference, where the AE reads its representations through one-way attention. Rather than prescribing what Predictive Intent should encode, PINT learns it jointly from the native prediction objectives of the VLM, VGM, and AE, which provide complementary views of the same upcoming task execution. Training uses two stages: Stage 1 freezes the VGM so that future video prediction shapes Predictive Intent with task-relevant future dynamics, while Stage 2 fixes the learned Intent and adapts the VGM and AE. Without embodied pretraining, PINT achieves strong in-distribution (ID) performance and substantially improves out-of-distribution (OOD) generalization, reaching 15.5% on RoboTwin 2.0-Hard compared with 1.8% for Fast-WAM and 3.2% for StarVLA, and 79.5% on LIBERO-Plus compared with 51.5% and 75.0%, respectively. Across five real-world bimanual tasks, PINT reaches 68% ID and 24% OOD success, compared with 35%/9% for pretrained Motus and 67%/35% for pretrained $\pi_{0.5}$.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
