机器学习算法超参数行为的实证研究
Behavior of Hyper-Parameters for Selected Machine Learning Algorithms: An Empirical Investigation
摘要:
超参数(HP)是机器学习(ML)模型开发的重要组成部分,并且可以极大地影响性能。本文研究了三种算法的行为:极端梯度提升(XGB),随机森林(RF)和前馈神经网络(FFNN)与结构化数据。我们的实证研究考察了模型性能的定性行为,因为HP的变化,量化的重要性,每个HP不同的ML算法,和稳定的性能接近最佳区域。基于这些发现,我们提出了一套准则,通过减少搜索空间的高效HP调整。
英文摘要:
Hyper-parameters (HPs) are an important part of machine learning (ML) model development and can greatly influence performance. This paper studies their behavior for three algorithms: Extreme Gradient Boosting (XGB), Random Forest (RF), and Feedforward Neural Network (FFNN) with structured data. Our empirical investigation examines the qualitative behavior of model performance as the HPs vary, quantifies the importance of each HP for different ML algorithms, and stability of the performance near the optimal region. Based on the findings, we propose a set of guidelines for efficient HP tuning by reducing the search space.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。