线性动力系统DP问题的代数分解
Algebraic Decompositions of DP Problems with Linear Dynamics
摘要:
受有理典范形的启发,我们引入并分析了线性动态系统的动态规划(DP)问题的两种分解。具体来说,我们考虑有限和无限地平线DP问题,其中的动态是线性的,费用函数只取决于状态,状态空间是有限维的,但定义在一个任意的代数域。从自然分解的状态空间的子空间是不变的系统的线性变换下的直接直和,并假设费用函数表现出与此分解兼容的添加剂结构,我们从原始DP问题中提取两个不同的家庭较小的DP问题,每个与系统的原始状态空间的不变子空间上发展。我们建议,这些家庭中的每一个构成了分解的原始问题时,原始问题的最优策略和价值函数可以从家庭中的各个子问题的最优策略和价值函数重建。我们得到了这些分解在有限和无限视界情况下都存在的充要条件。我们还提出了一个易于验证的充分条件下,第一个分解存在,我们表明,第一个概念的分解一般是强于第二个。
英文摘要:
Inspired by rational canonical forms, we introduce and analyze two decompositions of dynamic programming (DP) problems for systems with linear dynamics. Specifically, we consider both finite and infinite horizon DP problems in which the dynamics are linear, the cost function depends only on the state, and the state-space is finite dimensional but defined over an arbitrary algebraic field. Starting from the natural decomposition of the state-space into the direct sum of subspaces that are invariant under the system's linear transformation, and assuming that the cost functions exhibit an additive structure compatible with this decomposition, we extract from the original DP problem two distinct families of smaller DP problems, each associated with a system evolving on an invariant subspace of the original state-space. We propose that each of these families constitutes a decomposition of the original problem when the optimal policy and value function of the original problem can be reconstructed from the optimal policies and value functions of the individual subproblems in the family. We derive necessary and sufficient conditions for these decompositions to exist both in the finite and infinite horizon cases. We also propose a readily verifiable sufficient condition under which the first decomposition exists, and we show that the first notion of decomposition is generally stronger than the second.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。