目标核规划：面向具有目标条件化选项的逻辑任务的线性可解非马尔可夫策略
Goal Kernel Planning: Linearly-Solvable Non-Markovian Policies for Logical Tasks with Goal-Conditioned Options
AI总结：
本文提出LS-GKDP框架，融合LMDP与Options框架，通过目标核在低维任务接地子空间中优化非马尔可夫子目标任务的元策略，并支持零样本任务迁移。
摘要：
在层次化规划领域，组合性、抽象和任务迁移对于设计能够以最大程度的表征复用来高效求解各种问题的算法至关重要。许多现实世界问题需要非马尔可夫策略来处理具有逻辑条件的复杂结构化任务，这往往会导致过于庞大的状态表征；因此需要高效的方法来分解这些问题并在任务之间复用结构。为此，我们提出了一个称为线性可解目标核动态规划（LS-GKDP）的组合框架，以应对求解具有排序约束的非马尔可夫布尔子目标任务的复杂性。LS-GKDP将线性可解马尔可夫决策过程（LMDP）形式体系与强化学习的Options框架相结合。LMDP可以作为主特征向量问题被高效求解，而options是具有终止条件的策略，用作时间上延展的动作；借助LS-GKDP，我们扩展LMDP以对逻辑任务的options进行控制。这包括将高维问题分解为每个目标的一组目标条件options，并构建一个目标核，即一个抽象转移核，它从一个option的初始状态跳跃到其终止状态，同时更新更高层的任务状态。我们展示了具有目标核的LMDP如何能够在由任务接地所定义的较低维子空间中高效优化元策略。Options还可以在超指数任务空间中重新映射到新问题，而无需大量重新计算，并且我们识别出解对任务接地不变的情形，从而允许零样本任务迁移。
英文摘要：
In the domain of hierarchical planning, compositionality, abstraction, and task transfer are crucial for designing algorithms that can efficiently solve a variety of problems with maximal representational reuse. Many real-world problems require non-Markovian policies to handle complex structured tasks with logical conditions, often leading to prohibitively large state representations; this requires efficient methods for breaking these problems down and reusing structure between tasks. To this end, we introduce a compositional framework called Linearly-Solvable Goal Kernel Dynamic Programming (LS-GKDP) to address the complexity of solving non-Markovian Boolean sub-goal tasks with ordering constraints. LS-GKDP combines the Linearly-Solvable Markov Decision Process (LMDP) formalism with the Options Framework of Reinforcement Learning. LMDPs can be efficiently solved as a principal eigenvector problem, and options are policies with termination conditions used as temporally extended actions; with LS-GKDP we expand LMDPs to control over options for logical tasks. This involves decomposing a high-dimensional problem down into a set of goal-condition options for each goal and constructing a goal kernel, which is an abstract transition kernel that jumps from an option's initial-states to its termination-states along with an update of the higher-level task-state. We show how an LMDP with a goal kernel enables the efficient optimization of meta-policies in a lower-dimensional subspace defined by the task grounding. Options can also be remapped to new problems within a super-exponential space of tasks without significant recomputation, and we identify cases where the solution is invariant to the task grounding, permitting zero-shot task transfer.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
