arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 0810.0877math.NAcs.LGcs.NA

蒙特卡洛优化的偏差-方差技术:CE方法的交叉验证

Bias-Variance Techniques for Monte Carlo Optimization: Cross-validation for the CE Method

Dev Rajnarayan, David Wolpert

AI总结:

本文利用偏差-方差权衡和交叉验证技术改进交叉熵(CE)方法在蒙特卡洛优化中的性能,并指出参数学习中的技术可推广至优化算法。

摘要:

本文在蒙特卡洛优化(MCO)和参数学习(PL)的广泛背景下考察了CE方法,后者是一种机器学习。一个众所周知的用于提高许多PL算法性能的总体原则是偏差-方差权衡。该权衡已被用于改进从蒙特卡洛积分估计到线性估计再到一般统计估计的PL算法。此外,如所述,MCO与PL密切相关。由于这种相似性,偏差-方差权衡影响MCO性能,正如它影响PL性能一样。在本文中,我们利用偏差-方差权衡来增强MCO算法的性能。我们使用交叉验证技术,一种基于偏差-方差权衡的技术,来显著改进交叉熵(CE)方法(一种MCO算法)的性能。在先前的工作中,我们已确认其他PL技术改进了其他MCO算法的性能。我们得出结论,PL中开创的许多技术可以研究作为改进一般MCO算法,特别是CE方法的方法。

英文摘要:

In this paper, we examine the CE method in the broad context of Monte Carlo Optimization (MCO) and Parametric Learning (PL), a type of machine learning. A well-known overarching principle used to improve the performance of many PL algorithms is the bias-variance tradeoff. This tradeoff has been used to improve PL algorithms ranging from Monte Carlo estimation of integrals, to linear estimation, to general statistical estimation. Moreover, as described by, MCO is very closely related to PL. Owing to this similarity, the bias-variance tradeoff affects MCO performance, just as it does PL performance. In this article, we exploit the bias-variance tradeoff to enhance the performance of MCO algorithms. We use the technique of cross-validation, a technique based on the bias-variance tradeoff, to significantly improve the performance of the Cross Entropy (CE) method, which is an MCO algorithm. In previous work we have confirmed that other PL techniques improve the perfomance of other MCO algorithms. We conclude that the many techniques pioneered in PL could be investigated as ways to improve MCO algorithms in general, and the CE method in particular.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑