arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1909.09528math.OCmath.PRmath.STstat.MLstat.TH
收藏

脉冲控制问题的非参数学习

Nonparametric learning for impulse control problems

Sören Christensen, Claudia Strauch

摘要:

连续时间过程的随机控制的基本假设之一是,基本(扩散)过程的动力学是已知的。然而,这在实践中通常显然得不到满足。另一方面,在过去的几十年里,一个丰富的理论,连续时间过程的漂移(和波动率)的非参数估计已经发展。本文的目的是将随机控制的技术与随机过程的统计方法结合起来,找到一种既能了解潜在过程的动态又能同时以合理的方式进行控制的方法。更确切地说,我们研究了一个长期的平均脉冲控制问题,一个随机版本的经典福斯特曼木材采伐问题。立即出现的问题之一是探索-利用困境,这是机器学习中众所周知的问题。我们提出了一种方法来处理这个问题,以适当的方式结合勘探和开采期。我们的主要发现是,这种结构可以基于不变密度估计的收敛速度。利用这一点,我们得到了平均累积后悔是一致阶的$O({T^{-1/3}})$ .

英文摘要:

One of the fundamental assumptions in stochastic control of continuous time processes is that the dynamics of the underlying (diffusion) process is known. This is, however, usually obviously not fulfilled in practice. On the other hand, over the last decades, a rich theory for nonparametric estimation of the drift (and volatility) for continuous time processes has been developed. The aim of this paper is bringing together techniques from stochastic control with methods from statistics for stochastic processes to find a way to both learn the dynamics of the underlying process and control in a reasonable way at the same time. More precisely, we study a long-term average impulse control problem, a stochastic version of the classical Faustmann timber harvesting problem. One of the problems that immediately arises is an exploration-exploitation dilemma as is well known for problems in machine learning. We propose a way to deal with this issue by combining exploration and exploitation periods in a suitable way. Our main finding is that this construction can be based on the rates of convergence of estimators for the invariant density. Using this, we obtain that the average cumulated regret is of uniform order $O({T^{-1/3}})$.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑