Decision Transformer作为部分可观测连续控制基础模型的研究
Decision Transformer as a Foundation Model for Partially Observable Continuous Control
AI总结：
本文提出以GPT初始化并用LoRA训练的Decision Transformer框架，将部分可观测控制转化为序列动作预测，在五类任务上展现零样本泛化和少样本超越专家的能力。
摘要：
具有部分状态可观测性的非线性动力系统闭环控制，需要掌握一系列多样且标准化程度较低的理论工具的专业知识。此外，它还要求对控制器和估计器设计进行精细集成，以实现预期的系统行为。为建立通用的控制器综合框架，我们探索了Decision Transformer（DT）架构。具体而言，我们首先将控制任务表述为基于过去观测、动作和奖励预测当前最优动作，从而无需单独设计估计器。随后，我们利用预训练语言模型，即Generative Pre-trained Transformer（GPT）系列，来初始化DT，并使用低秩适配（LoRA）针对控制任务进行训练。我们在五个不同控制任务上开展了全面实验，范围从航空航天系统机动控制到偏微分方程（PDEs）控制，结果表明DT能够捕捉控制任务内在的、与参数无关的结构。DT对全新任务表现出显著的零样本泛化能力，并能借助极少量演示数据迅速超越专家性能水平。这些发现凸显了DT作为通用控制应用基础控制器的潜力。
英文摘要：
Closed-loop control of nonlinear dynamical systems with partial-state observability demands expert knowledge of a diverse, less standardized set of theoretical tools. Moreover, it requires a delicate integration of controller and estimator designs to achieve the desired system behavior. To establish a general controller synthesis framework, we explore the Decision Transformer (DT) architecture. Specifically, we first frame the control task as predicting the current optimal action based on past observations, actions, and rewards, eliminating the need for a separate estimator design. Then, we leverage the pre-trained language models, i.e., the Generative Pre-trained Transformer (GPT) series, to initialize DT and subsequently train it for control tasks using low-rank adaptation (LoRA). Our comprehensive experiments across five distinct control tasks, ranging from maneuvering aerospace systems to controlling partial differential equations (PDEs), demonstrate DT's capability to capture the parameter-agnostic structures intrinsic to control tasks. DT exhibits remarkable zero-shot generalization abilities for completely new tasks and rapidly surpasses expert performance levels with a minimal amount of demonstration data. These findings highlight the potential of DT as a foundational controller for general control applications.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
