随机估计中偏倚-方差权衡的改进平衡：极大极小观点
Enhanced Balancing of Bias-Variance Tradeoff in Stochastic Estimation: A Minimax Perspective
摘要：
有偏的随机估计，如噪声梯度估计的有限差分，通常包含需要适当选择的参数，以平衡偏差和方差的影响。虽然这些参数在模拟预算方面的最佳顺序可以很容易地建立，但精确的最佳值取决于通常事先未知的模型特性。我们引入了一个框架来构建新的类的估计，基于明智的组合的模拟运行序列的调整参数值，这样的估计始终优于给定的调整参数的选择，在传统的方法，无论未知的模型特性。我们认为，通过我们所谓的渐近极小极大风险比，通过最小化最坏情况下的渐近比之间的均方误差的估计和传统的，其中最坏的情况下是超过任何可能的值的模型未知数的表现。特别地，当极小极大比小于1时，保证校准的估计器在渐近上执行得更好。我们确定这个极小极大比一般类的加权估计，和政权，这个比例小于1。此外，我们表明，最好的加权方案的特点是由两个组件具有不同的衰减率的总和。我们解释了这是如何从偏差-方差平衡中产生的，该平衡对抗了模型常数的对抗性选择，这可以通过非凸优化问题的易处理的重新表述来分析。
英文摘要：
Biased stochastic estimators, such as finite-differences for noisy gradient estimation, often contain parameters that need to be properly chosen to balance impacts from the bias and the variance. While the optimal order of these parameters in terms of the simulation budget can be readily established, the precise best values depend on model characteristics that are typically unknown in advance. We introduce a framework to construct new classes of estimators, based on judicious combinations of simulation runs on sequences of tuning parameter values, such that the estimators consistently outperform a given tuning parameter choice in the conventional approach, regardless of the unknown model characteristics. We argue the outperformance via what we call the asymptotic minimax risk ratio, obtained by minimizing the worst-case asymptotic ratio between the mean square errors of our estimators and the conventional one, where the worst case is over any possible values of the model unknowns. In particular, when the minimax ratio is less than 1, the calibrated estimator is guaranteed to perform better asymptotically. We identify this minimax ratio for general classes of weighted estimators, and the regimes where this ratio is less than 1. Moreover, we show that the best weighting scheme is characterized by a sum of two components with distinct decay rates. We explain how this arises from bias-variance balancing that combats the adversarial selection of the model constants, which can be analyzed via a tractable reformulation of a non-convex optimization problem.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
