面向结构分解马尔可夫决策过程的基于因果解缠的改进蒙特卡洛规划
Improved Monte Carlo Planning via Causal Disentanglement for Structurally-Decomposed Markov Decision Processes
AI总结：
针对传统MDP未利用因果结构的问题，提出基于因果解缠的SD-MDP框架，将序列优化转化为对数线性复杂度的分数背包问题，可与MCTS集成，在多领域策略性能优于基准。
摘要：
马尔可夫决策过程（MDPs）作为通用框架，往往忽略了引入转移和奖励动力学因果结构的益处。针对一类资源分配问题子类，我们提出了结构分解MDP（SD-MDP），该模型利用因果解缠将MDP的时间因果图划分为独立组件。通过利用这种解缠特性，SD-MDP可在最优值函数估计中实现降维并提升计算效率。我们将序列优化问题转化为分数背包问题，其复杂度为对数线性级$O(T \log T)$，优于时间跨度$T$下呈多项式复杂度的传统随机规划方法。此外，SD-MDP的计算优势与状态-动作空间规模无关，因此适用于高维空间。进一步地，我们的方法可与蒙特卡洛树搜索（MCTS）无缝集成，在有限模拟预算下获得更高的期望奖励，同时具备趋近于零的简单遗憾界。实证结果表明，在物流和金融等多个领域，该方法的策略性能优于基准方法。
英文摘要：
Markov Decision Processes (MDPs), as a general-purpose framework, often overlook the benefits of incorporating the causal structure of the transition and reward dynamics. For a subclass of resource allocation problems, we introduce the Structurally Decomposed MDP (SD-MDP), which leverages causal disentanglement to partition an MDP's temporal causal graph into independent components. By exploiting this disentanglement, SD-MDP enables dimensionality reduction and computational efficiency gains in optimal value function estimation. We reduce the sequential optimization problem to a fractional knapsack problem with log-linear complexity $O(T \log T)$, outperforming traditional stochastic programming methods that exhibit polynomial complexity with respect to the time horizon $T$. Additionally, SD-MDP's computational advantages are independent of state-action space size, making it viable for high-dimensional spaces. Furthermore, our approach integrates seamlessly with Monte Carlo Tree Search (MCTS), achieving higher expected rewards under constrained simulation budgets while providing a vanishing simple regret bound. Empirical results demonstrate superior policy performance over benchmarks across various logistics and finance domains.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
