arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1607.00372cs.PFcs.LO
收藏

基于策略迭代的固定时延CTMC系统的有效综合

Efficient Timeout Synthesis in Fixed-Delay CTMC Using Policy Iteration

Ľuboš Korenčiak, Antonín Kučera, Vojtěch Řehák

摘要:

本文研究了具有固定延迟转移的连续时间马尔可夫链的固定延迟综合问题。我们的目标是合成固定延迟(超时)的具体值,使达到给定的一组目标状态之前产生的预期总成本最小化。同样的问题已经被认为是解决在以前的作品中计算的最优策略,在一定的离散时间马尔可夫决策过程(MDP)与大量的行动,对应于适当的离散值的超时。在本文中,我们设计了一个符号固定延迟综合算法,避免了显式构造大的行动空间。相反,该算法根据需要计算一小部分“有希望的”候选动作。通过计算其符号导数并提取一个一元多项式来最小化某个目标函数,从而选择候选动作,该一元多项式的根恰好是导数取零值的点。由于高次一元多项式的根可以非常有效地隔离使用现代数学软件,我们不仅实现了大幅节省内存,但也加速了三个数量级相比,以前的方法。

英文摘要:

We consider the fixed-delay synthesis problem for continuous-time Markov chains extended with fixed-delay transitions (fdCTMC). The goal is to synthesize concrete values of the fixed-delays (timeouts) that minimize the expected total cost incurred before reaching a given set of target states. The same problem has been considered and solved in previous works by computing an optimal policy in a certain discrete-time Markov decision process (MDP) with a huge number of actions that correspond to suitably discretized values of the timeouts. In this paper, we design a symbolic fixed-delay synthesis algorithm which avoids the explicit construction of large action spaces. Instead, the algorithm computes a small sets of "promising" candidate actions on demand. The candidate actions are selected by minimizing a certain objective function by computing its symbolic derivative and extracting a univariate polynomial whose roots are precisely the points where the derivative takes zero value. Since roots of high degree univariate polynomials can be isolated very efficiently using modern mathematical software, we achieve not only drastic memory savings but also speedup by three orders of magnitude compared to the previous methods.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑