双向解码:通过引导式测试时采样改进动作分块
Bidirectional Decoding: Improving Action Chunking via Guided Test-Time Sampling
AI总结:
本文提出双向解码(BID)算法,通过向后连贯性和向前对比标准在测试时采样最优动作,解决了机器人学习中动作分块带来的长期一致性与短期响应性之间的权衡,显著提升了多种策略在仿真和真实任务中的性能。
摘要:
预测并执行一系列动作而无需中间重新规划,这被称为动作分块(action chunking),它在从人类示范中学习机器人操作的应用日益增多。然而,它对学习到的策略的影响仍然不一致:一些研究发现它对取得优异结果至关重要,而另一些研究则观察到性能下降。在本文中,我们首先剖析了动作分块如何影响学习者与示范者之间的分歧。我们发现,动作分块使学习者能够更好地捕捉示范中的时间依赖性,但代价是降低了对意外状态的响应能力。为了解决这一权衡问题,我们提出了双向解码(BID),这是一种将动作分块与闭环适应相结合的测试时推理算法。在每个时间步,BID采样多个候选预测,并基于两个标准搜索最优预测:(i) 向后连贯性,倾向于与先前决策保持一致的样本;(ii) 向前对比,寻找未来计划具有高可能性的样本。通过在动作分块内部和跨分块之间耦合决策,BID促进了长期一致性和短期响应能力。实验结果表明,我们的方法在七个仿真基准和两个真实世界任务中,提升了两种最先进的生成式策略的性能。代码和视频可在 https://bid-robot.github.io 获取。
英文摘要:
Predicting and executing a sequence of actions without intermediate replanning, known as action chunking, is increasingly used in robot learning from human demonstrations. Yet, its effects on the learned policy remain inconsistent: some studies find it crucial for achieving strong results, while others observe decreased performance. In this paper, we first dissect how action chunking impacts the divergence between a learner and a demonstrator. We find that action chunking allows the learner to better capture the temporal dependencies in demonstrations but at the cost of reduced reactivity to unexpected states. To address this tradeoff, we propose Bidirectional Decoding (BID), a test-time inference algorithm that bridges action chunking with closed-loop adaptation. At each timestep, BID samples multiple candidate predictions and searches for the optimal one based on two criteria: (i) backward coherence, which favors samples that align with previous decisions; (ii) forward contrast, which seeks samples of high likelihood for future plans. By coupling decisions within and across action chunks, BID promotes both long-term consistency and short-term reactivity. Experimental results show that our method boosts the performance of two state-of-the-art generative policies across seven simulation benchmarks and two real-world tasks. Code and videos are available at https://bid-robot.github.io.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。