面向连续时间延迟系统的神经拉普拉斯控制
Neural Laplace Control for Continuous-time Delayed Systems
AI总结：
本文提出 Neural Laplace Control，将 Neural Laplace 动力学模型与 MPC 规划器结合，解决连续时间环境中非规则观测与未知延迟并存的离线强化学习问题，并实验证明其接近专家策略性能。
摘要：
许多现实世界的离线强化学习（RL）问题涉及具有延迟的连续时间环境。这类环境具有两个显著特征：首先，状态 x(t) 在非规则时间间隔被观测；其次，当前动作 a(t) 仅以未知延迟 g > 0 影响未来状态 x(t + g)。此类环境的一个典型例子是卫星控制，其中地球与卫星之间的通信链路导致非规则观测和延迟。现有的离线 RL 算法已在时间上非规则观测状态或已知延迟的环境中取得成功。然而，同时涉及时间上非规则观测和未知延迟的环境仍然是一个开放且具有挑战性的问题。为此，我们提出 Neural Laplace Control，一种基于模型的连续时间离线 RL 方法，它将 Neural Laplace 动力学模型与模型预测控制（MPC）规划器相结合——并且能够从以非规则时间间隔采样的离线数据集中学习，该数据集来自具有固有未知恒定延迟的环境。我们在连续时间延迟环境中的实验表明，该方法能够达到接近专家策略的性能。
英文摘要：
Many real-world offline reinforcement learning (RL) problems involve continuous-time environments with delays. Such environments are characterized by two distinctive features: firstly, the state x(t) is observed at irregular time intervals, and secondly, the current action a(t) only affects the future state x(t + g) with an unknown delay g > 0. A prime example of such an environment is satellite control where the communication link between earth and a satellite causes irregular observations and delays. Existing offline RL algorithms have achieved success in environments with irregularly observed states in time or known delays. However, environments involving both irregular observations in time and unknown delays remains an open and challenging problem. To this end, we propose Neural Laplace Control, a continuous-time model-based offline RL method that combines a Neural Laplace dynamics model with a model predictive control (MPC) planner--and is able to learn from an offline dataset sampled with irregular time intervals from an environment that has a inherent unknown constant delay. We show experimentally on continuous-time delayed environments it is able to achieve near expert policy performance.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
