The false coverage rate (FCR) is the expected ratio of number of constructed confidence intervals (CIs) that fail to cover their respective parameters to the total number of constructed CIs. Procedures for FCR control exist in the offline setting, but none so far have been designed with the online setting in mind. In the online setting, there is an infinite sequence of fixed unknown parameters $θ_t$ ordered by time. At each step, we see independent data that is informative about $θ_t$, and must immediately make a decision whether to report a CI for $θ_t$ or not. If $θ_t$ is selected for coverage, the task is to determine how to construct a CI for $θ_t$ such that $\text{FCR} \leq α$ for any $T\in \mathbb{N}$. A straightforward solution is to construct at each step a $(1-α)$ level conditional CI. In this paper, we present a novel solution to the problem inspired by online false discovery rate (FDR) algorithms, which only requires the statistician to be able to construct a marginal CI at any given level. Apart from the fact that marginal CIs are usually simpler to construct than conditional ones, the marginal procedure has an important qualitative advantage over the conditional solution, namely, it allows selection to be determined by the candidate CI itself. We take advantage of this to offer solutions to some online problems which have not been addressed before. For example, we show that our general CI procedure can be used to devise online sign-classification procedures that control the false sign rate (FSR). In terms of power and length of the constructed CIs, we demonstrate that the two approaches have complementary strengths and weaknesses using simulations. Last, all of our methodology applies equally well to online FCR control for prediction intervals, having particular implications for assumption-free selective conformal inference.
MMDCP: A Distribution-free Approach to Outlier Detection and Classification with Coverage Guarantees and SCW-FDR Control
Guanghua School of Management, Peking University(北京大学光华管理学院);School of Mathematics and Computing Science, Guilin University of Electronic Technology(桂林电子科技大学数学与计算机学院);College of Energy and Power Engineering, Nanjing University of Aeronautics and Astronautics(南京航空航天大学能源与动力学院);School of Business, Guangdong University of Foreign Studies(广东外语外贸大学商学院)