arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2412.09265cs.ROcs.LGstat.ML

Score and Distribution Matching Policy:通过匹配蒸馏的先进加速视觉运动策略

Score and Distribution Matching Policy: Advanced Accelerated Visuomotor Policies via Matched Distillation

Westlake University(西湖大学);Southwest University(西南大学);Zhejiang University(浙江大学);Beijing Advanced Innovation Center for Future Blockchain and Privacy Computing(北京未来区块链与隐私计算高精尖创新中心);School of Artificial Intelligence, Beihang University(北京航空航天大学人工智能学院)

Bofang Jia, Pengxiang Ding, Can Cui, Mingyang Sun, Pengfang Qian, Siteng Huang, Zhaoxin Fan, Donglin Wang

AI总结:

针对扩散策略推理慢和一致性蒸馏误差问题,提出SDM Policy,通过score matching与distribution matching两阶段优化及双教师机制,将扩散策略转为单步生成器,实现6倍加速并保持最先进动作质量。

摘要:

视觉运动策略学习随着基于扩散的策略等架构而进步,这些架构以建模复杂机器人轨迹著称。然而,其漫长的推理时间阻碍了需要实时反馈的高频控制任务。虽然一致性蒸馏(CD)加速了推理,但会引入损害动作质量的误差。为解决这些限制,我们提出Score and Distribution Matching Policy(SDM Policy),通过两阶段优化将基于扩散的策略转化为单步生成器:score matching确保与真实动作分布对齐,distribution matching最小化KL散度以保持一致性。双教师机制整合冻结教师以确保稳定性与未冻结教师进行对抗训练,增强鲁棒性与目标分布对齐。在57任务模拟基准上评估,SDM Policy实现6倍推理加速并具最先进动作质量,为高频机器人任务提供高效可靠框架。

英文摘要:

Visual-motor policy learning has advanced with architectures like diffusion-based policies, known for modeling complex robotic trajectories. However, their prolonged inference times hinder high-frequency control tasks requiring real-time feedback. While consistency distillation (CD) accelerates inference, it introduces errors that compromise action quality. To address these limitations, we propose the Score and Distribution Matching Policy (SDM Policy), which transforms diffusion-based policies into single-step generators through a two-stage optimization process: score matching ensures alignment with true action distributions, and distribution matching minimizes KL divergence for consistency. A dual-teacher mechanism integrates a frozen teacher for stability and an unfrozen teacher for adversarial training, enhancing robustness and alignment with target distributions. Evaluated on a 57-task simulation benchmark, SDM Policy achieves a 6x inference speedup while having state-of-the-art action quality, providing an efficient and reliable framework for high-frequency robotic tasks.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑