arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2201.00879cs.LGeess.SP
收藏

基于Actor-Critic的受控感知用于异常时序检测

Temporal Detection of Anomalies via Actor-Critic Based Controlled Sensing

Syracuse University(雪城大学)

Geethu Joseph, M. Cenk Gursoy, Pramod K. Varshney

AI总结:

针对二元随机过程异常数超阈值的时序检测问题,提出基于贝叶斯后验概率与深度actor-critic强化学习的三步序贯受控感知决策方法,性能优于传统基于模型的算法。

摘要:

我们研究了监控一组二元随机过程,并在其中异常数量超过阈值时发出警报的问题。为此,决策者选择并探测部分过程,以获取其状态(正常或异常)的含噪估计。基于接收到的观测值,决策者首先判断是宣布异常数量已超过阈值,还是继续采集观测值。若决定继续,则进一步决定是在下一时刻采集观测值,还是推迟到后续时刻。若选择采集观测值,则还要确定待探测的过程子集。为设计这一三步序贯决策流程,我们采用贝叶斯公式,学习过程状态的后验概率。基于该后验概率,我们构建了马尔可夫决策过程,并使用深度actor-critic强化学习求解。通过数值实验,我们证明了该算法相比传统基于模型的算法具有更优性能。

英文摘要:

We address the problem of monitoring a set of binary stochastic processes and generating an alert when the number of anomalies among them exceeds a threshold. For this, the decision-maker selects and probes a subset of the processes to obtain noisy estimates of their states (normal or anomalous). Based on the received observations, the decisionmaker first determines whether to declare that the number of anomalies has exceeded the threshold or to continue taking observations. When the decision is to continue, it then decides whether to collect observations at the next time instant or defer it to a later time. If it chooses to collect observations, it further determines the subset of processes to be probed. To devise this three-step sequential decision-making process, we use a Bayesian formulation wherein we learn the posterior probability on the states of the processes. Using the posterior probability, we construct a Markov decision process and solve it using deep actor-critic reinforcement learning. Via numerical experiments, we demonstrate the superior performance of our algorithm compared to the traditional model-based algorithms.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑