arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2610.23334cs.RO
收藏

LAW:统一语言、动作与世界建模的具身物理智能框架

LAW: Unifying Language, Action, and World Modeling for Agentic Physical Intelligence

Jiajun Cao, Kai Tang, Bohua Zhang, Zezhong Qian, Ying Li, Xiaozhu Ju, Xiaobao Wei, Jiaming Liu, Yang Ding, Sixiang Chen, Yiding Sun, Siyuan Qian, Tinghao Wang, … 展开作者

Jiajun Cao, Kai Tang, Bohua Zhang, Zezhong Qian, Ying Li, Xiaozhu Ju, Xiaobao Wei, Jiaming Liu, Yang Ding, Sixiang Chen, Yiding Sun, Siyuan Qian, Tinghao Wang, Jixian Wu, Yichen Guo, Yi Zhang, Shilong Zou, Zhiyuan qin, Yong Dai, Jian Tang, Shanghang Zhang

AI总结:

本文提出统一语言、动作与世界建模的闭环VLM-WAM框架LAW,在RoboTwin任务上达94.21%平均成功率,替换主干后升至95.56%,在真实机器人长程任务中也优于基线模型。

摘要:

世界-动作模型可联合预测动作与未来世界状态,但长程操纵任务还需语言推理来规划子目标、评估执行进度并指导动作选择。本文提出LAW,这是一个通过语义子任务进度统一语言推理、动作生成与世界建模的闭环VLM-WAM框架。在LAW智能体内部,混合专家视觉-语言模型规划剩余子任务,估计并融合世界-动作模型提出的动作的值,异步评估执行状态以支持持续执行与子任务转换;在世界-动作模型内部,Transformer专家混合模型整合模态特定与共享计算,用于联合视频-动作建模。双流分层视觉记忆为两个模型维护时间上下文,而子任务局部值学习使动作评估与语义目标对齐,并保留失败轨迹成功前缀的监督信号。LAW在50个RoboTwin任务上实现94.21%的平均成功率,将WAM主干替换为OpenWAM-α后,平均成功率进一步提升至95.56%,在RoboTwin上建立了新的开源最优水平。尽管仅含2B参数,本文的VLM在性能上优于使用Embodied-R1.5和GPT-5.5 High的变体;在3个长程真实机器人任务中,LAW实现65%的平均成功率,而π₀.₅为28.33%,Fast-WAM为25%。

英文摘要:

World-action models jointly predict actions and future world states, but long-horizon manipulation further requires language reasoning to plan subtask objectives, assess execution progress, and guide action selection. We present LAW, a closed-loop VLM--WAM framework that unifies language reasoning, action generation, and world modeling through semantic subtask progress. Within the LAW agent, a hybrid-expert vision-language model plans the remaining subtasks, estimates the values of and fuses actions proposed by a world-action model, and asynchronously assesses execution status to support continued execution and subtask transitions. Within the world-action model, Mixture-of-Transformer Experts integrates modality-specific and shared computation for joint video--action modeling. Dual-stream hierarchical visual memory maintains temporal context for both models, while subtask-local value learning aligns action evaluation with semantic objectives and preserves supervision from successful prefixes of failed trajectories. LAW achieves 94.21% average success across 50 RoboTwin tasks. Replacing the WAM backbone with OpenWAM-$alpha$ further raises the average success rate to 95.56%, establishing a new open-source state of the art on RoboTwin. Despite having only 2B parameters, our VLM also outperforms variants using Embodied-R1.5 and GPT-5.5 High. On three long-horizon real-robot tasks, LAW achieves 65% average success, compared with 28.33% for $pi_{0.5}$ and 25% for Fast-WAM.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑