FRAPPE:通过多未来表示对齐将世界建模注入通用策略
FRAPPE: Infusing World Modeling into Generalist Policies via Multiple Future Representation Alignment
AI总结:
FRAPPE通过多未来表示对齐提升通用机器人策略的世界建模能力,提高微调效率并减少数据依赖,实验证明其在长时序和未见场景中的泛化性能优于现有方法。
摘要:
使VLA模型能够预测环境动态,即世界建模,已被认为是提高机器人推理和泛化能力的关键。然而,当前方法面临两个主要问题:1. 训练目标迫使模型过度强调像素级重建,这限制了语义学习和泛化能力;2. 推理过程中依赖预测的未来观察往往导致误差累积。为了解决这些挑战,我们引入了通过并行逐步扩展的未来表示对齐(FRAPPE)。我们的方法采用两阶段微调策略:在中期训练阶段,模型学习预测未来观察的潜在表示;在后期训练阶段,我们并行扩展计算负载,并同时与多种不同的视觉基础模型对齐表示。通过显著提高微调效率并减少对动作标注数据的依赖,FRAPPE提供了一条可扩展且数据高效的途径,以增强通用机器人策略中的世界意识。在RoboTwin基准和现实任务中的实验表明,FRAPPE优于现有最先进的方法,并在长时间跨度和未见场景中表现出强大的泛化能力。
英文摘要:
Enabling VLA models to predict environmental dynamics, known as world modeling, has been recognized as essential for improving robotic reasoning and generalization. However, current approaches face two main issues: 1. The training objective forces models to over-emphasize pixel-level reconstruction, which constrains semantic learning and generalization 2. Reliance on predicted future observations during inference often leads to error accumulation. To address these challenges, we introduce Future Representation Alignment via Parallel Progressive Expansion (FRAPPE). Our method adopts a two-stage fine-tuning strategy: In the mid-training phase, the model learns to predict the latent representations of future observations; In the post-training phase, we expand the computational workload in parallel and align the representation simultaneously with multiple different visual foundation models. By significantly improving fine-tuning efficiency and reducing dependence on action-annotated data, FRAPPE provides a scalable and data-efficient pathway to enhance world-awareness in generalist robotic policies. Experiments on the RoboTwin benchmark and real-world tasks demonstrate that FRAPPE outperforms state-of-the-art approaches and shows strong generalization in long-horizon and unseen scenarios.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。