In linear regression we wish to estimate the optimum linear least squares predictor for a distribution over $d$-dimensional input points and real-valued responses, based on a small sample. Under standard random design analysis, where the sample is drawn i.i.d. from the input distribution, the least squares solution for that sample can be viewed as the natural estimator of the optimum. Unfortunately, this estimator almost always incurs an undesirable bias coming from the randomness of the input points, which is a significant bottleneck in model averaging. In this paper we show that it is possible to draw a non-i.i.d. sample of input points such that, regardless of the response model, the least squares solution is an unbiased estimator of the optimum. Moreover, this sample can be produced efficiently by augmenting a previously drawn i.i.d. sample with an additional set of $d$ points, drawn jointly according to a certain determinantal point process constructed from the input distribution rescaled by the squared volume spanned by the points. Motivated by this, we develop a theoretical framework for studying volume-rescaled sampling, and in the process prove a number of new matrix expectation identities. We use them to show that for any input distribution and $ε>0$ there is a random design consisting of $O(d\log d+ d/ε)$ points from which an unbiased estimator can be constructed whose expected square loss over the entire distribution is bounded by $1+ε$ times the loss of the optimum. We provide efficient algorithms for generating such unbiased estimators in a number of practical settings and support our claims experimentally.
Unbiased estimators for the variance of MMD estimators
最大平均差异（MMD）是在许多应用中有用的概率分布之间的基于核的距离（Gretton et al. 2012），具有令人满意的计算和统计特性的简单估计量。能够有效地估计该估计量的方差，对解决双样本检验中的各种问题都有很大的帮助。为此，Bounliphone等人。（2016）使用U-统计量理论来推导MMD估计量方差的估计量，以及两个这样的估计量之间的差异。然而，他们的估计器丢弃了低阶项，并且不必要地存在偏差。我们在这篇笔记中表明-扩展和纠正Sutherland等人的工作。（2017）-我们可以找到一个真正的无偏估计量，用于平方MMD估计量和两个相关平方MMD估计量之差的实际方差，基本上没有额外的计算成本。
Unbiased and Biased Variance-Reduced Forward-Reflected-Backward Splitting Methods for Stochastic Composite Inclusions
Department of Statistics and Operations Research（统计与运筹学系）；The University of North Carolina at Chapel Hill（北卡罗来纳大学教堂山分校）；Hanes Hall, UNC-Chapel Hill, NC 27599-3260（UNC-教堂山分校318号哈尼斯楼，NC 27599-3260）
