In this paper, we develop a general approach for probabilistic estimation and optimization. An explicit formula and a computational approach are established for controlling the reliability of probabilistic estimation based on a mixed criterion of absolute and relative errors. By employing the Chernoff-Hoeffding bound and the concept of sampling, the minimization of a probabilistic function is transformed into an optimization problem amenable for gradient descendent algorithms.
In this paper we derive sharp lower and upper bounds for the covariance of two bounded random variables when knowledge about their expected values, variances or both is available. When only the expected values are known, our result can be viewed as an extension of the Bhatia-Davis Inequality for variances. We also provide a number of different ways to standardize covariance. For a binary pair random variables, one of these standardized measures of covariation agrees with a frequently used measure of dependence between genetic variants.
College of Information and Computer Sciences（信息与计算机科学学院）；University of Massachusetts（马萨诸塞大学）
We present a new method for constructing a confidence interval for the mean of a bounded random variable from samples of the random variable. We conjecture that the confidence interval has guaranteed coverage, i.e., that it contains the mean with high probability for all distributions on a bounded interval, for all samples sizes, and for all confidence levels. This new method provides confidence intervals that are competitive with those produced using Student's t-statistic, but does not rely on normality assumptions. In particular, its only requirement is that the distribution be bounded on a …
We investigate the sub-Gaussian property for almost surely bounded random variables. If sub-Gaussianity per se is de facto ensured by the bounded support of said random variables, then exciting research avenues remain open. Among these questions is how to characterize the optimal sub-Gaussian proxy variance? Another question is how to characterize strict sub-Gaussianity, defined by a proxy variance equal to the (standard) variance? We address the questions in proposing conditions based on the study of functions variations. A particular focus is given to the relationship between strict sub-Gaus…
We study an unbiased estimator for the density of a sum of random variables that are simulated from a computer model. A numerical study on examples with copula dependence is conducted where the proposed estimator performs favourably in terms of variance compared to other unbiased estimators. We provide applications and extensions to the estimation of marginal densities in Bayesian statistics and to the estimation of the density of sums of random variables under Gaussian copula dependence.
We study the estimation, in Lp-norm, of density functions defined on [0,1]^d. We construct a new family of kernel density estimators that do not suffer from the so-called boundary bias problem and we propose a data-driven procedure based on the Goldenshluger and Lepski approach that jointly selects a kernel and a bandwidth. We derive two estimators that satisfy oracle-type inequalities. They are also proved to be adaptive over a scale of anisotropic or isotropic Sobolev-Slobodetskii classes (which are particular cases of Besov or Sobolev classical classes). The main interest of the isotropic p…
We derive the optimal signed variable in general case kernels for the classical statistic density estimation, which are some generalization of the famous Epanechnikov's ones.
We study the mutual information estimation for mixed-pair random variables. One random variable is discrete and the other one is continuous. We develop a kernel method to estimate the mutual information between the two random variables. The estimates enjoy a central limit theorem under some regular conditions on the distributions. The theoretical results are demonstrated by simulation study.
Factor models are a parsimonious way to explain the dependence of variables using several latent variables. In Gaussian 1-factor and structural factor models (such as bi-factor, oblique factor) and their factor copula counterparts, factor scores or proxies are defined as conditional expectations of latent variables given the observed variables. With mild assumptions, the proxies are consistent for corresponding latent variables as the sample size and the number of observed variables linked to each latent variable go to infinity. When the bivariate copulas linking observed variables to latent v…
Many inference problems, such as sequential decision problems like A/B testing, adaptive sampling schemes like bandit selection, are often online in nature. The fundamental problem for online inference is to provide a sequence of confidence intervals that are valid uniformly over the growing-into-infinity sample sizes. To address this question, we provide a near-optimal confidence sequence for bounded random variables by utilizing Bentkus' concentration results. We show that it improves on the existing approaches that use the Cram{é}r-Chernoff technique such as the Hoeffding, Bernstein, and Be…
Shannon entropy is widely used to quantify the uncertainty of discrete random variables. But when normalized to the unit interval, as is often done in practice, it no longer conveys the alphabet sizes of the random variables being studied. This work introduces an entropy functional based on Jensen-Shannon divergence that is naturally bounded from above by one. Unlike normalized Shannon entropy, this new functional is strictly increasing in alphabet size under uniformity and is thus well suited to the characterization of discrete random variables.
Most studies for negatively associated (NA) random variables consider the complete-data situation, which is actually a relatively ideal condition in practice. The paper relaxes this condition to the incomplete-data setting and considers kernel smoothing density and hazard function estimation in the presence of right censoring based on the Kaplan-Meier estimator. We establish the strong asymptotic properties for these two estimators to assess their asymptotic behavior and justify their practical use.
Maximum-likelihood estimation (MLE) is arguably the most important tool for statisticians, and many methods have been developed to find the MLE. We present a new inequality involving posterior distributions of a latent variable that holds under very general conditions. It is related to the EM algorithm and has a clear potential for being used in a similar fashion.
Context Tree Estimation in Variable Length Hidden Markov Models
我们解决了可变长度隐马尔可夫模型中的上下文树估计问题。我们提出了一个估计的上下文树的隐马尔可夫过程，它不需要事先上界的深度上下文树。我们证明了估计量是强相合的。这一方法使用了Finesso和Lorenzo（Consistent estimation of the order for Markov and hidden Markov chains（1990））和E.Gassiat和S.Boucheron（Optimal error exponents in hidden Markov model order estimation（2003））的信息论混合不等式。我们提出了一种算法来有效地计算估计，并提供模拟研究，以支持我们的结果。
This paper derives confidence intervals (CI) and time-uniform confidence sequences (CS) for the classical problem of estimating an unknown mean from bounded observations. We present a general approach for deriving concentration bounds, that can be seen as a generalization and improvement of the celebrated Chernoff method. At its heart, it is based on a class of composite nonnegative martingales, with strong connections to testing by betting and the method of mixtures. We show how to extend these ideas to sampling without replacement, another heavily studied problem. In all cases, our bounds ar…
