arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2511.04421cs.RO
收藏

用于动作分块的时间动作选择

Temporal Action Selection for Action Chunking

Guangdong Key Laboratory of Intelligent Morphing Mechanisms and Adaptive Robotics and School of Intelligence Science and Engineering, the Harbin Institute of Technology Shenzhen, China(广东省智能变形机制与自适应机器人重点实验室和智能科学与工程学院,哈尔滨工业大学深圳学院)

Yueyang Weng, Xiaopeng Zhang, Yongjin Mu, Yingcong Zhu, Yanjie Li

AI总结:

提出时间动作选择(TAS)算法,通过缓存多时间步预测的动作块并动态选择最优动作,在保持决策一致性的同时提升反应性,显著提高任务成功率。

摘要:

动作分块是学习从示范(LfD)中广泛采用的方法。通过建模多步动作块而非单步动作,动作分块显著增强了对人类专家策略的建模能力。然而,由于动作分块仅在完整动作块执行后才做出单一决策,由此导致的决策频率降低限制了实时观测的利用,削弱了在动态或嘈杂环境中的反应性。现有解决该问题的尝试主要是在反应性和决策一致性之间进行权衡,未能同时实现两者。为解决这一局限,我们提出了一种新颖算法——时间动作选择(TAS),该算法缓存来自多个时间步的预测动作块,并通过轻量级选择器网络动态选择最优动作。TAS在反应性和决策一致性上实现了平衡优化。跨多个任务及不同基础策略架构的实验表明,TAS显著提高了成功率。此外,将TAS作为基础策略与残差强化学习(RL)相结合,既提升了训练效率,也提高了性能上限。在仿真和物理机器人上的实验均证实了该方法的有效性。

英文摘要:

Action chunking is a widely adopted approach in Learning from Demonstration (LfD). By modeling multi-step action chunks rather than single-step actions, action chunking significantly enhances modeling capabilities for human expert policies. However, because action chunking makes a single decision only after a complete action block has been executed, the resulting reduction in decision frequency restricts the utilization of real-time observations, impairing reactivity in dynamic or noisy environments. Existing efforts to address this issue have primarily resorted to trading off reactivity against decision consistency, without achieving both. To address this limitation, we propose a novel algorithm, Temporal Action Selection (TAS), which caches predicted action chunks from multiple timesteps and dynamically selects the optimal action through a lightweight selector network. TAS achieves balanced optimization across both reactivity and decision consistency. Experiments across multiple tasks with diverse base policy architectures show that TAS significantly improves success rates. Furthermore, integrating TAS as a base policy with residual reinforcement learning (RL) improves both training efficiency and the performance ceiling. Experiments in both simulation and physical robots confirm the method's efficacy.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑