超越随机分裂的统计模型性能评估
Beyond Random Split for Assessing Statistical Model Performance
摘要：
尽管随机执行的数据集的训练/测试分割是一种常见的做法，但在某些情况下，它并不总是估计性能泛化的最佳方法。事实上，当数据集不具有代表性或者当罕见和难以捉摸的示例是检测问题的基本方面时，通常的机器学习方法有时会高估泛化误差。在目前的工作中，我们分析了基于预测变量的可变性的策略，以在训练集和测试集中进行分割。这种策略的目的是保证包括罕见或不寻常的例子与人口的代表性损失最小，并提供一个更准确的估计时，数据集是不具有代表性的泛化误差。基于决策树的两个基线分类器被用于测试所考虑的四个分裂策略。这两种分类器都应用于CTU 19，一个低代表性的数据集的网络安全检测问题。初步结果表明，应用三种替代策略的Monte Carlo分裂策略，以获得更准确的误差估计不同的，但可行的情况下的重要性。
英文摘要：
Even though a train/test split of the dataset randomly performed is a common practice, could not always be the best approach for estimating performance generalization under some scenarios. The fact is that the usual machine learning methodology can sometimes overestimate the generalization error when a dataset is not representative or when rare and elusive examples are a fundamental aspect of the detection problem. In the present work, we analyze strategies based on the predictors' variability to split in training and testing sets. Such strategies aim at guaranteeing the inclusion of rare or unusual examples with a minimal loss of the population's representativeness and provide a more accurate estimation about the generalization error when the dataset is not representative. Two baseline classifiers based on decision trees were used for testing the four splitting strategies considered. Both classifiers were applied on CTU19 a low-representative dataset for a network security detection problem. Preliminary results showed the importance of applying the three alternative strategies to the Monte Carlo splitting strategy in order to get a more accurate error estimation on different but feasible scenarios.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
