arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2402.06187cs.LGcs.AIcs.RO

Premier-TACO 是一个少样本策略学习者:通过时间动作驱动对比损失预训练多任务表示

Premier-TACO is a Few-Shot Policy Learner: Pretraining Multitask Representation via Temporal Action-Driven Contrastive Loss

University of Maryland(马里兰大学);Microsoft Research(微软研究院);Tsinghua University(清华大学)

Ruijie Zheng, Yongyuan Liang, Xiyao Wang, Shuang Ma, Hal Daumé, Huazhe Xu, John Langford, Praveen Palanisamy, Kalyan Shankar Basu, Furong Huang

AI总结:

Premier-TACO 通过改进时间动作对比学习并引入新负样本采样策略,利用多任务离线数据预训练视觉表示,显著提升连续控制任务中的少样本策略学习效率。

摘要:

我们提出了 Premier-TACO,一种多任务特征表示学习方法,旨在提高序贯决策任务中的少样本策略学习效率。Premier-TACO 利用多任务离线数据集的一个子集来预训练通用特征表示,该表示捕获关键的环境动态,并通过最少的专家演示进行微调。它推进了时间动作对比学习(TACO)目标(该目标在视觉控制任务中取得了最先进的结果),通过引入一种新颖的负样本采样策略。这一策略对于显著提升 TACO 的计算效率至关重要,使得大规模多任务离线预训练成为可能。我们在包括 Deepmind Control Suite、MetaWorld 和 LIBERO 在内的多样化连续控制基准上的广泛实证评估证明了 Premier-TACO 在预训练视觉表示方面的有效性,显著增强了新任务的少样本模仿学习。我们的代码、预训练数据以及预训练模型检查点将在 https://github.com/PremierTACO/premier-taco 发布。我们的项目网页位于 https://premiertaco.github.io。

英文摘要:

We present Premier-TACO, a multitask feature representation learning approach designed to improve few-shot policy learning efficiency in sequential decision-making tasks. Premier-TACO leverages a subset of multitask offline datasets for pretraining a general feature representation, which captures critical environmental dynamics and is fine-tuned using minimal expert demonstrations. It advances the temporal action contrastive learning (TACO) objective, known for state-of-the-art results in visual control tasks, by incorporating a novel negative example sampling strategy. This strategy is crucial in significantly boosting TACO's computational efficiency, making large-scale multitask offline pretraining feasible. Our extensive empirical evaluation in a diverse set of continuous control benchmarks including Deepmind Control Suite, MetaWorld, and LIBERO demonstrate Premier-TACO's effectiveness in pretraining visual representations, significantly enhancing few-shot imitation learning of novel tasks. Our code, pretraining data, as well as pretrained model checkpoints will be released at https://github.com/PremierTACO/premier-taco. Our project webpage is at https://premiertaco.github.io.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑