LAW:统一语言、动作与世界建模的具身物理智能框架
LAW: Unifying Language, Action, and World Modeling for Agentic Physical Intelligence
AI总结:
本文提出统一语言、动作与世界建模的闭环VLM-WAM框架LAW,在RoboTwin任务上达94.21%平均成功率,替换主干后升至95.56%,在真实机器人长程任务中也优于基线模型。
摘要:
世界-动作模型可联合预测动作与未来世界状态,但长程操纵任务还需语言推理来规划子目标、评估执行进度并指导动作选择。本文提出LAW,这是一个通过语义子任务进度统一语言推理、动作生成与世界建模的闭环VLM-WAM框架。在LAW智能体内部,混合专家视觉-语言模型规划剩余子任务,估计并融合世界-动作模型提出的动作的值,异步评估执行状态以支持持续执行与子任务转换;在世界-动作模型内部,Transformer专家混合模型整合模态特定与共享计算,用于联合视频-动作建模。双流分层视觉记忆为两个模型维护时间上下文,而子任务局部值学习使动作评估与语义目标对齐,并保留失败轨迹成功前缀的监督信号。LAW在50个RoboTwin任务上实现94.21%的平均成功率,将WAM主干替换为OpenWAM-α后,平均成功率进一步提升至95.56%,在RoboTwin上建立了新的开源最优水平。尽管仅含2B参数,本文的VLM在性能上优于使用Embodied-R1.5和GPT-5.5 High的变体;在3个长程真实机器人任务中,LAW实现65%的平均成功率,而π₀.₅为28.33%,Fast-WAM为25%。
英文摘要:
World-action models jointly predict actions and future world states, but long-horizon manipulation further requires language reasoning to plan subtask objectives, assess execution progress, and guide action selection. We present LAW, a closed-loop VLM--WAM framework that unifies language reasoning, action generation, and world modeling through semantic subtask progress. Within the LAW agent, a hybrid-expert vision-language model plans the remaining subtasks, estimates the values of and fuses actions proposed by a world-action model, and asynchronously assesses execution status to support continued execution and subtask transitions. Within the world-action model, Mixture-of-Transformer Experts integrates modality-specific and shared computation for joint video--action modeling. Dual-stream hierarchical visual memory maintains temporal context for both models, while subtask-local value learning aligns action evaluation with semantic objectives and preserves supervision from successful prefixes of failed trajectories. LAW achieves 94.21% average success across 50 RoboTwin tasks. Replacing the WAM backbone with OpenWAM-$alpha$ further raises the average success rate to 95.56%, establishing a new open-source state of the art on RoboTwin. Despite having only 2B parameters, our VLM also outperforms variants using Embodied-R1.5 and GPT-5.5 High. On three long-horizon real-robot tasks, LAW achieves 65% average success, compared with 28.33% for $pi_{0.5}$ and 25% for Fast-WAM.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。