Statistically Model Checking PCTL Specifications on Markov Decision Processes via Reinforcement Learning

Lifted Model Checking for Relational MDPs

Probabilistic model checking has been developed for verifying systems that have stochastic and nondeterministic behavior. Given a probabilistic system, a probabilistic model checker takes a property and checks whether or not the property holds in that system. For this reason, probabilistic model checking provide rigorous guarantees. So far, however, probabilistic model checking has focused on propositional models where a state is represented by a symbol. On the other hand, it is commonly required to make relational abstractions in planning and reinforcement learning. Various frameworks handle …

An Abstraction-based Method to Check Multi-Agent Deep Reinforcement-Learning Behaviors

Multi-agent reinforcement learning (RL) often struggles to ensure the safe behaviours of the learning agents, and therefore it is generally not adapted to safety-critical applications. To address this issue, we present a methodology that combines formal verification with (deep) RL algorithms to guarantee the satisfaction of formally-specified safety constraints both in training and testing. The approach we propose expresses the constraints to verify in Probabilistic Computation Tree Logic (PCTL) and builds an abstract representation of the system to reduce the complexity of the verification st…

How Many Random Seeds? Statistical Power Analysis in Deep Reinforcement Learning Experiments

Consistently checking the statistical significance of experimental results is one of the mandatory methodological steps to address the so-called "reproducibility crisis" in deep reinforcement learning. In this tutorial paper, we explain how the number of random seeds relates to the probabilities of statistical errors. For both the t-test and the bootstrap confidence interval test, we recall theoretical guidelines to determine the number of random seeds one should use to provide a statistically significant comparison of the performance of two algorithms. Finally, we discuss the influence of dev…

COOL-MC：验证和解释血小板库存管理中的强化学习策略

COOL-MC: Verifying and Explaining RL Policies for Platelet Inventory Management

COOL-MC通过结合RL与概率模型检验，验证并解释了血小板库存管理中强化学习策略的可靠性与安全性。

弱潜在因子何时能被统计推断？

When can weak latent factors be statistically inferred?

本文在允许截面相关特质成分的弱因子模型下建立了PCA的全面非渐近估计与推断理论，证明信噪比仅需以log N的多项式速率增长即可保证渐近正态性，并将理论应用于因子验证、结构突变检验和风险暴露比较等问题。

Pseudo Random Number Generation: a Reinforcement Learning approach

Pseudo-Random Numbers Generators (PRNGs) are algorithms produced to generate long sequences of statistically uncorrelated numbers, i.e. Pseudo-Random Numbers (PRNs). These numbers are widely employed in mid-level cryptography and in software applications. Test suites are used to evaluate PRNGs quality by checking statistical properties of the generated sequences. Machine learning techniques are often used to break these generators, for instance approximating a certain generator or a certain sequence using a neural network. But what about using machine learning to generate PRNs generators? This…