不要丢弃尾部：通过动作升级加速策略
Don't Throw Away the Tail: Action Upcycling for Policy Acceleration
浏览论文内容
中文总结 AI 辅助
提出动作升级算法，通过重用被丢弃的动作延长执行视界，无需额外采样或访问模型内部，将策略调用减少1.2-1.7倍且不损失成功率。
中文摘要 AI 辅助
现代机器人策略从单次观测中预测未来动作的一个块，仅执行前缀部分，并在重新规划前丢弃其余部分。选择此前缀的长度，即执行视界，需要在反应性和效率之间进行权衡。短视界使策略对环境保持反应性，但需要频繁调用策略。最近的测试时方法为每个块自适应选择视界，但它们要么读取模型内部信息（此时必须为每种架构选择信号），要么额外采样（这增加了成本）。我们提出动作升级（Action Upcycling），一种无需训练且不访问模型内部或额外采样的算法，它重用策略原本会丢弃的动作。我们发现，只要动作速度保持平滑，被丢弃的动作与重新规划后的版本保持接近。因此，动作升级将执行视界延长至速度开始波动的点。在模拟和真实世界操作任务上的大量实验表明，动作升级将策略调用次数减少了1.2--1.7倍，且成功率无损失，适用于多种视觉-语言-动作模型（VLA）甚至世界动作模型（WAM）。它适用于任何分块策略，成本可忽略，并且与其他策略加速方法（如少步采样和流式动作解码）正交，为策略加速开辟了新的维度。
英文摘要
Modern robot policies predict a chunk of future actions from a single observation, execute only a prefix, and discard the rest before replanning. Choosing the length of this prefix, the execution horizon, poses a trade-off between reactivity and efficiency. A short horizon keeps the policy reactive to the environment, but requires frequent policy calls. Recent test-time methods adaptively select the horizon for each chunk, but they either read model internals, where the signal must be chosen for each architecture, or draw extra samples, which adds cost. We propose Action Upcycling, a training-free algorithm that reuses actions the policy would otherwise discard, without accessing model internals or drawing extra samples. We find that discarded actions stay close to their replanned versions as long as the action velocity remains smooth. Action Upcycling therefore extends the execution horizon up to the point where the velocity begins to fluctuate. Extensive experiments on simulated and real-world manipulation tasks show that Action Upcycling reduces policy calls by 1.2-1.7x with no loss in success rate, across multiple Vision-Language-Action Models (VLAs) and even a World Action Model (WAM). It applies to any chunked policy at negligible cost and is orthogonal to other policy acceleration methods such as few-step sampling and streaming action decoding, opening a new axis for policy acceleration.
发表机构
- KAIST（韩国科学技术院）
- Sungkyunkwan University（成均馆大学）
机构由 AI 辅助整理，请以论文原文为准。
