arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2510.01068cs.ROcs.LG
收藏

制定你的策略!通过测试时的分布级组合改进扩散型或流型机器人策略

Compose Your Policies! Improving Diffusion-based or Flow-based Robot Policies via Test-time Distribution-level Composition

The University of Hong Kong(香港大学);Beijing Innovation Center of Humanoid Robotics(北京人形机器人创新中心);Shanghai AI Lab(上海人工智能实验室);Shanghai Jiaotong University(上海交通大学);The Hong Kong University of Science and Technology(香港科技大学)
Jiahang Cao, Yize Huang, Hanzhong Guo, Rui Zhang, Mu Nan, Weijian Mai, Jiaxu Wang, Hao Cheng, Jingkai Sun, Gang Han, Wen Zhao, Qiang Zhang, Yijie Guo, Qihao Zhe… 展开作者

Jiahang Cao, Yize Huang, Hanzhong Guo, Rui Zhang, Mu Nan, Weijian Mai, Jiaxu Wang, Hao Cheng, Jingkai Sun, Gang Han, Wen Zhao, Qiang Zhang, Yijie Guo, Qihao Zheng, Chunfeng Song, Xiao Li, Ping Luo, Andrew F. Luo

AI总结:

本研究提出无需训练的通用策略组合方法,通过测试时分布级组合提升机器人策略性能。

摘要:

基于扩散的机器人控制模型,包括视觉-语言-动作(VLA)和视觉-动作(VA)策略,已展现出显著的能力。然而,其进步受到获取大规模交互数据集成本高的限制。本工作引入了一种替代范式,以在不增加模型训练的情况下提高策略性能。出人意料的是,我们证明组合策略可以超越任一父策略的性能。我们的贡献有三方面。首先,我们建立了理论基础,证明多个扩散模型的分布评分的凸组合可以产生优于任何单个评分的一步功能目标。随后,通过Grönwall型界证明,这种单步改进会通过整个生成轨迹传播,导致系统性的性能提升。第二,受这些结果启发,我们提出了通用策略组合(GPC),一种无需训练的方法,通过凸组合多个预训练策略的分布评分并在测试时进行搜索来提高性能。GPC具有灵活性,允许将异构策略进行即插即用的组合,包括VA和VLA模型,以及基于扩散或流匹配的策略,无论其输入的视觉模态如何。第三,我们提供了广泛的实证验证。在Robomimic、PushT和RoboTwin基准测试,以及现实世界的机器人评估中,实验确认GPC在多样化的任务集上一致提高了性能和适应性。对替代组合运算符和加权策略的进一步分析提供了对GPC成功机制的见解。这些结果确立了GPC作为一种简单而有效的改进控制性能的方法,通过利用现有策略。

英文摘要:

Diffusion-based models for robotic control, including vision-language-action (VLA) and vision-action (VA) policies, have demonstrated significant capabilities. Yet their advancement is constrained by the high cost of acquiring large-scale interaction datasets. This work introduces an alternative paradigm for enhancing policy performance without additional model training. Perhaps surprisingly, we demonstrate that the composed policies can exceed the performance of either parent policy. Our contribution is threefold. First, we establish a theoretical foundation showing that the convex composition of distributional scores from multiple diffusion models can yield a superior one-step functional objective compared to any individual score. A Grönwall-type bound is then used to show that this single-step improvement propagates through entire generation trajectories, leading to systemic performance gains. Second, motivated by these results, we propose General Policy Composition (GPC), a training-free method that enhances performance by combining the distributional scores of multiple pre-trained policies via a convex combination and test-time search. GPC is versatile, allowing for the plug-and-play composition of heterogeneous policies, including VA and VLA models, as well as those based on diffusion or flow-matching, irrespective of their input visual modalities. Third, we provide extensive empirical validation. Experiments on Robomimic, PushT, and RoboTwin benchmarks, alongside real-world robotic evaluations, confirm that GPC consistently improves performance and adaptability across a diverse set of tasks. Further analysis of alternative composition operators and weighting strategies offers insights into the mechanisms underlying the success of GPC. These results establish GPC as a simple yet effective method for improving control performance by leveraging existing policies.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑