思维链预测控制
Chain-of-Thought Predictive Control
AI总结：
本文提出 CoTPC，通过无监督发现演示的子技能思维链，并用 Transformer 联合预测子技能与动作，在次优演示的复杂操作任务中提升策略泛化性能。
摘要：
我们研究面向复杂低层控制（例如，富接触物体操作）的、可从演示中泛化的策略学习。我们提出一种利用次优演示的新型分层模仿学习方法。首先，我们提出一种与观测空间无关的方法，以无监督方式高效发现演示的多步子技能分解。通过将时间上接近且功能上相似的动作分组为子技能级演示片段，片段边界处的观测构成该任务的规划步骤链，我们称之为思维链（CoT）。接下来，我们提出一种基于 Transformer 的设计，有效学习预测 CoT 作为子技能级指导。我们通过可学习提示 token 和混合掩码策略耦合动作预测与子技能预测，这使得测试时能够动态更新指导，并改善轨迹的特征表示，以实现可泛化的策略学习。我们的方法 Chain-of-Thought Predictive Control（CoTPC，思维链预测控制）在具有次优演示的挑战性操作任务上，持续超越现有强基线。
英文摘要：
We study generalizable policy learning from demonstrations for complex low-level control (e.g., contact-rich object manipulations). We propose a novel hierarchical imitation learning method that utilizes sub-optimal demos. Firstly, we propose an observation space-agnostic approach that efficiently discovers the multi-step subskill decomposition of the demos in an unsupervised manner. By grouping temporarily close and functionally similar actions into subskill-level demo segments, the observations at the segment boundaries constitute a chain of planning steps for the task, which we refer to as the chain-of-thought (CoT). Next, we propose a Transformer-based design that effectively learns to predict the CoT as the subskill-level guidance. We couple action and subskill predictions via learnable prompt tokens and a hybrid masking strategy, which enable dynamically updated guidance at test time and improve feature representation of the trajectory for generalizable policy learning. Our method, Chain-of-Thought Predictive Control (CoTPC), consistently surpasses existing strong baselines on challenging manipulation tasks with sub-optimal demos.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
