arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2110.12403cs.LGstat.ML
收藏

学习无偏估计

Learning to Estimate Without Bias

The Hebrew University of Jerusalem(希伯来大学);Weizmann Institute of Science(魏茨曼科学研究所)

Tzvi Diskin, Yonina C. Eldar, Ami Wiesel

AI总结:

本文提出偏置约束估计器(BCE),通过在深度学习损失函数中加入偏置约束,在非线性场景下实现渐近最小方差无偏估计,并验证其在信噪比估计、协方差估计及多估计平均应用中的有效性。

摘要:

高斯-马尔可夫定理指出,加权最小二乘估计器是线性模型中的线性最小方差无偏估计(MVUE)。在本文中,我们通过带有偏置约束的深度学习,向将该结果扩展到非线性场景迈出了第一步。设计非线性MVUE的经典方法是通过最大似然估计(MLE),这通常涉及计算上具有挑战性的优化问题。另一方面,深度学习方法允许以固定的计算复杂度构建非线性估计器。基于学习的估计器在其训练集上平均表现最优,但在其他参数上可能存在显著偏置。为避免这一问题,我们提出在损失函数中加入一个简单的偏置约束,从而得到我们称之为偏置约束估计器(BCE)的估计器。我们证明,该估计器能够产生渐近MVUE,其行为与经典MLE相似,并渐近达到克拉美-罗界。我们在信噪比估计以及协方差估计的背景下展示了所提方法的优势。BCE的第二个动机在于需要对同一未知量的多个估计进行平均以提高性能的应用中。此类应用的例子包括分布式传感器网络和测试时的数据增强。在这些应用中,我们证明BCE能够产生渐近一致的估计器。

英文摘要:

The Gauss Markov theorem states that the weighted least squares estimator is a linear minimum variance unbiased estimation (MVUE) in linear models. In this paper, we take a first step towards extending this result to non linear settings via deep learning with bias constraints. The classical approach to designing non-linear MVUEs is through maximum likelihood estimation (MLE) which often involves computationally challenging optimizations. On the other hand, deep learning methods allow for non-linear estimators with fixed computational complexity. Learning based estimators perform optimally on average with respect to their training set but may suffer from significant bias in other parameters. To avoid this, we propose to add a simple bias constraint to the loss function, resulting in an estimator we refer to as Bias Constrained Estimator (BCE). We prove that this yields asymptotic MVUEs that behave similarly to the classical MLEs and asymptotically attain the Cramer Rao bound. We demonstrate the advantages of our approach in the context of signal to noise ratio estimation as well as covariance estimation. A second motivation to BCE is in applications where multiple estimates of the same unknown are averaged for improved performance. Examples include distributed sensor networks and data augmentation in test-time. In such applications, we show that BCE leads to asymptotically consistent estimators.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑