PIVOT-R：用于机器人操作的基元驱动航点感知世界模型
PIVOT-R: Primitive-Driven Waypoint-Aware World Model for Robotic Manipulation
AI总结：
针对语言引导机器人操作中模型易受动态环境变化影响的问题，提出PIVOT-R，通过航点感知世界模型和异步分层执行器，在SeaWave基准上实现19.45%的平均相对提升，并将执行效率提高28倍。
摘要：
语言引导的机器人操作是一项具有挑战性的任务，要求具身智能体遵循抽象的用户指令来完成各种复杂的操作任务。以往的工作只是简单地拟合数据，而没有揭示指令与低层可执行动作之间的关系，这些模型容易记住数据的表面模式，而不是获得可迁移的知识，因此面对动态环境变化时很脆弱。为了解决这个问题，我们提出了一种用于机器人操作的基元驱动航点感知世界模型（PIVOT-R），它仅专注于预测与任务相关的航点。具体来说，PIVOT-R由一个航点感知世界模型（WAWM）和一个轻量级动作预测模块组成。前者执行基元动作解析和基元驱动的航点预测，而后者专注于解码低层动作。此外，我们还设计了一个异步分层执行器（AHE），它可以为模型的不同模块使用不同的执行频率，从而帮助模型减少计算冗余并提高模型执行效率。我们的PIVOT-R在SeaWave基准上优于最先进的开源模型，在四个级别的指令任务中实现了19.45%的平均相对提升。此外，与同步执行的PIVOT-R相比，带有AHE的PIVOT-R的执行效率提高了28倍，而性能仅下降2.9%。这些结果提供了令人信服的证据，表明我们的PIVOT-R能够显著提升机器人操作的性能和效率。
英文摘要：
Language-guided robotic manipulation is a challenging task that requires an embodied agent to follow abstract user instructions to accomplish various complex manipulation tasks. Previous work trivially fitting the data without revealing the relation between instruction and low-level executable actions, these models are prone to memorizing the surficial pattern of the data instead of acquiring the transferable knowledge, and thus are fragile to dynamic environment changes. To address this issue, we propose a PrIrmitive-driVen waypOinT-aware world model for Robotic manipulation (PIVOT-R) that focuses solely on the prediction of task-relevant waypoints. Specifically, PIVOT-R consists of a Waypoint-aware World Model (WAWM) and a lightweight action prediction module. The former performs primitive action parsing and primitive-driven waypoint prediction, while the latter focuses on decoding low-level actions. Additionally, we also design an asynchronous hierarchical executor (AHE), which can use different execution frequencies for different modules of the model, thereby helping the model reduce computational redundancy and improve model execution efficiency. Our PIVOT-R outperforms state-of-the-art (SoTA) open-source models on the SeaWave benchmark, achieving an average relative improvement of 19.45% across four levels of instruction tasks. Moreover, compared to the synchronously executed PIVOT-R, the execution efficiency of PIVOT-R with AHE is increased by 28-fold, with only a 2.9% drop in performance. These results provide compelling evidence that our PIVOT-R can significantly improve both the performance and efficiency of robotic manipulation.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
