arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2402.10450cs.LG
收藏

PRISE:用于控制中学习时间动作抽象的LLM风格序列压缩

PRISE: LLM-Style Sequence Compression for Learning Temporal Action Abstractions in Control

University of Maryland(马里兰大学);Microsoft(微软公司)

Ruijie Zheng, Ching-An Cheng, Hal Daumé, Furong Huang, Andrey Kolobov

AI总结:

PRISE将连续动作量化与BPE结合,把时间动作抽象学习视为序列压缩问题,从多任务机器人演示中提取高层技能,显著提升多任务和少样本模仿学习性能。

摘要:

时间动作抽象与信念状态表示一起,是顺序决策中一种强大的知识共享机制。在这项工作中,我们提出了一种新观点,将归纳时间动作抽象视为序列压缩问题。为此,我们将LLM训练流程中一个微妙但关键的组成部分——通过字节对编码(BPE)进行输入分词——引入到看似遥远的连续控制领域中学习可变时间跨度技能的任务中。我们提出了一种称为Primitive Sequence Encoding(PRISE)的方法,该方法将连续动作量化与BPE相结合,以学习强大的动作抽象。我们通过实验表明,PRISE从多任务机器人操作演示集中发现的高层技能显著提升了多任务模仿学习以及在未见任务上的少样本模仿学习的性能。我们的代码发布在https://github.com/FrankZheng2022/PRISE。

英文摘要:

Temporal action abstractions, along with belief state representations, are a powerful knowledge sharing mechanism for sequential decision making. In this work, we propose a novel view that treats inducing temporal action abstractions as a sequence compression problem. To do so, we bring a subtle but critical component of LLM training pipelines -- input tokenization via byte pair encoding (BPE) -- to the seemingly distant task of learning skills of variable time span in continuous control domains. We introduce an approach called Primitive Sequence Encoding (PRISE) that combines continuous action quantization with BPE to learn powerful action abstractions. We empirically show that high-level skills discovered by PRISE from a multitask set of robotic manipulation demonstrations significantly boost the performance of both multitask imitation learning as well as few-shot imitation learning on unseen tasks. Our code is released at https://github.com/FrankZheng2022/PRISE.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑