通过游戏抽象实现增量数据驱动的策略合成
Incremental Data-Driven Policy Synthesis via Game Abstractions
AI总结:
本文提出一种基于数据驱动和抽象的控制框架,通过增量游戏求解方法,在系统动态数据积累时逐步构建抽象游戏图、获胜区域和控制策略,实现对未知离散时间随机动态系统的控制策略合成。
摘要:
我们针对未知离散时间随机动态系统控制策略的合成问题,提出了一个数据驱动、基于抽象的控制框架,该框架整合了在线学习与新颖的增量游戏求解方法。在适当的连续性假设下,我们的方法将系统动态抽象为一个有限的随机(2.5玩家)游戏图,该图来源于数据。给定对时间的要求,我们计算该图的获胜区域——即初始状态集合,从这些状态出发可以满足目标。我们的主要贡献是逐步构建抽象、获胜区域和控制策略,随着系统动态数据的积累。具体而言,我们的算法在新数据样本到来时,逐步细化每个状态-动作对的可达集的下界和上界估计。这些细化导致游戏图抽象中的结构变化——例如节点和边的添加或删除——从而改变获胜区域。关键的是,我们证明这些更新本质上是单调的:下界估计只会增长,上界估计只会缩小,而获胜区域只会扩大。我们利用这种单调性,定义了一个基于目标诱导的节点排名函数,在抽象游戏中对节点进行排名,随着新数据样本的加入,该排名单调递增。这些排名支撑了我们新颖的增量游戏求解算法,该算法在排名提升算法中使用定制的子游戏(类似DAG的子游戏)来高效更新获胜区域。数值案例研究显示,与基线方法相比,该方法在新数据样本到来时重新求解整个游戏的计算开销显著减少。
英文摘要:
We address the synthesis of control policies for unknown discrete-time stochastic dynamical systems to satisfy temporal logic objectives. We present a data-driven, abstraction-based control framework that integrates online learning with novel incremental game-solving. Under appropriate continuity assumptions, our method abstracts the system dynamics into a finite stochastic (2.5-player) game graph derived from data. Given a requirement over time on this graph, we compute the winning region -- i.e., the set of initial states from which the objective is satisfiable -- in the resulting game, together with a corresponding control policy. Our main contribution is the construction of abstractions, winning regions and control policies \emph{incrementally}, as data about the system dynamics accumulates. Concretely, our algorithm refines under- and over-approximations of reachable sets for each state-action pair as new data samples arrive. These refinements induce structural modifications in the game graph abstraction -- such as the addition or removal of nodes and edges -- which in turn modify the winning region. Crucially, we show that these updates are inherently monotonic: under-approximations only grow, over-approximations only shrink, and the winning region only expands. We exploit this monotonicity by defining an objective-induced ranking function on the nodes of the abstract game that increases monotonically as new data samples are incorporated. These ranks underpin our novel incremental game-solving algorithm, which employs customized gadgets (DAG-like subgames) within a rank-lifting algorithm to efficiently update the winning region. Numerical case studies demonstrate significant computational savings compared to the baseline approach, which re-solves the entire game from scratch whenever new data samples arrive.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。