Sparse parametric models are of great interest in statistical learning and are often analyzed by means of regularized estimators. Pathwise methods allow to efficiently compute the full solution path for penalized estimators, for any possible value of the penalization parameter $λ$. In this paper we deal with the pathwise optimization for bridge-type problems; i.e. we are interested in the minimization of a loss function, such as negative log-likelihood or residual sum of squares, plus the sum of $\ell^q$ norms with $q\in(0,1]$ involving adpative coefficients. For some loss functions this regularization achieves asymptotically the oracle properties (such as the selection consistency). Nevertheless, since the objective function involves nonconvex and nondifferentiable terms, the minimization problem is computationally challenging.
The aim of this paper is to apply some general algorithms, arising from nonconvex optimization theory, to compute efficiently the path solutions for the adaptive bridge estimator with multiple penalties. In particular, we take into account two different approaches: accelerated proximal gradient descent and blockwise alternating optimization. The convergence and the path consistency of these algorithms are discussed. In order to assess our methods, we apply these algorithms to the penalized estimation of diffusion processes observed at discrete times. This latter represents a recent research topic in the field of statistics for time-dependent data.
Online monotone density estimation and log-optimal calibration
Department of Statistics and Data Science, Carnegie Mellon University, USA（统计与数据科学系，卡内基梅隆大学，美国）；Department of Statistics and Actuarial Science, University of Waterloo, Canada（统计与精算科学系，滑铁卢大学，加拿大）
Statistical Analysis of the Sinkhorn Iterations for Two-Sample Schrödinger Bridge Estimation
Department of Artificial Intelligence, Kyushu Institute of Technology（九州工学院人工智能系）；Department of Mathematics, University of British Columbia（不列颠哥伦比亚大学数学系）；CFAR and IHPC, Agency for Science, Technology and Research (A ⋆ \star STAR)（科技研究局（A ⋆ \star STAR））；College of Computing and Data Science, Nanyang Technological University（南洋理工大学计算与数据科学学院）
Bridge sampling is a powerful Monte Carlo method for estimating ratios of normalizing constants. Various methods have been introduced to improve its efficiency. These methods aim to increase the overlap between the densities by applying appropriate transformations to them without changing their normalizing constants. In this paper, we first give a new estimator of the asymptotic relative mean square error (RMSE) of the optimal Bridge estimator by equivalently estimating an $f$-divergence between the two densities. We then utilize this framework and propose $f$-GAN-Bridge estimator ($f$-GB) bas…
Columbia University（哥伦比亚大学）；Michigan State university（密歇根州立大学）
A recently proposed SLOPE estimator (arXiv:1407.3824) has been shown to adaptively achieve the minimax $\ell_2$ estimation rate under high-dimensional sparse linear regression models (arXiv:1503.08393). Such minimax optimality holds in the regime where the sparsity level $k$, sample size $n$, and dimension $p$ satisfy $k/p \rightarrow 0$, $k\log p/n \rightarrow 0$. In this paper, we characterize the estimation error of SLOPE under the complementary regime where both $k$ and $n$ scale linearly with $p$, and provide new insights into the performance of SLOPE estimators. We first derive a concent…
Technical University of Denmark（丹麦技术大学）；University of Copenhagen（哥本哈根大学）；Rigshospitalet, Copenhagen University Hospital（哥本哈根大学医院国立医院）
This paper introduces novel results for the score function gradient estimator of the importance weighted variational bound (IWAE). We prove that in the limit of large $K$ (number of importance samples) one can choose the control variate such that the Signal-to-Noise ratio (SNR) of the estimator grows as $\sqrt{K}$. This is in contrast to the standard pathwise gradient estimator where the SNR decreases as $1/\sqrt{K}$. Based on our theoretical findings we develop a novel control variate that extends on VIMCO. Empirically, for the training of both continuous and discrete generative models, the p…
InfoBridge: Mutual Information estimation via Bridge Matching
Applied AI Institute（应用人工智能研究所）；Institute of Numerical Mathematics (RAS)（俄罗斯科学院数值数学研究所）；Moscow Independent Research Institute of Artificial Intelligence（莫斯科独立人工智能研究所）
London School of Economics and Political Science（伦敦政治经济学院）；Cornell University（康奈尔大学）；University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）
We consider off-policy evaluation (OPE) in Partially Observable Markov Decision Processes (POMDPs), where the evaluation policy depends only on observable variables and the behavior policy depends on unobservable latent variables. Existing works either assume no unmeasured confounders, or focus on settings where both the observation and the state spaces are tabular. In this work, we first propose novel identification methods for OPE in POMDPs with latent confounders, by introducing bridge functions that link the target policy's value and the observed data distribution. We next propose minimax …
