技术报告：评价指标和抽样对生物多样性指标预测机器学习方法比较的影响
Technical report: Impact of evaluation metrics and sampling on the comparison of machine learning methods for biodiversity indicators prediction
摘要：
机器学习方法在生物多样性监测中的应用越来越广泛。特别地，一个重要的应用是基于预测器集合来预测生物多样性指标（诸如物种丰度、物种出现或物种丰富度）的问题，所述预测器集合包含，例如，气候和人为因素。考虑到文献中可用的不同ML方法的数量令人印象深刻，以及它们被出版的速度，制定统一的评估程序至关重要，以允许产生合理和公平的实证研究。然而，确定公平的评价程序是一项挑战：由于生物多样性指标的零膨胀和过度分散等内在特性有据可查，因此设计良好的交叉验证抽样方案和良好的评价指标并非易事。实际上，经典的均方误差（MSE）未能捕获不同方法的性能中的细微差异，特别是在预测非常小或非常大的值（例如，零计数或大计数）。在这份报告中，我们通过比较10个统计和机器学习模型来说明这一现象，这些模型基于地理，气象和时空因素来预测北非地区的水鸟丰度。我们的研究结果强调，不同的现成的评估指标和交叉验证抽样方法产生截然不同的排名的指标，并未能捕捉到可解释的结论。
英文摘要：
Machine learning (ML) approaches are used more and more widely in biodiversity monitoring. In particular, an important application is the problem of predicting biodiversity indicators such as species abundance, species occurrence or species richness, based on predictor sets containing, e.g., climatic and anthropogenic factors. Considering the impressive number of different ML methods available in the litterature and the pace at which they are being published, it is crucial to develop uniform evaluation procedures, to allow the production of sound and fair empirical studies. However, defining fair evaluation procedures is challenging: because well-documented, intrinsic properties of biodiversity indicators such as their zero-inflation and over-dispersion, it is not trivial to design good sampling schemes for cross-validation nor good evaluation metrics. Indeed, the classical Mean Squared Error (MSE) fails to capture subtle differences in the performance of different methods, particularly in terms of prediction of very small, or very large values (e.g., zero counts or large counts). In this report, we illustrate this phenomenon by comparing ten statistical and machine learning models on the task of predicting waterbirds abundance in the North-African area, based on geographical, meteorological and spatio-temporal factors. Our results highlight that differnte off-the-shelf evaluation metrics and cross-validation sampling approaches yield drastically different rankings of the metrics, and fail to capture interpretable conclusions.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
