arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2004.11123cs.LGstat.APstat.ML
收藏

大型传感器网络中缺失的亚小时降水数据的插补:一种机器学习方法

Imputation of missing sub-hourly precipitation data in a large sensor network: a machine learning approach

School of Computer Science, University of Lincoln(林肯大学计算机科学学院);UK Centre for Ecology and Hydrology(英国生态与水文中心);University of Aberdeen(阿伯丁大学)

Benedict Delahaye Chivers, John Wallbank, Steven J. Cole, Ondrej Sebek, Simon Stanley, Matthew Fry, Georgios Leontidis

摘要:

以亚小时分辨率收集的降水数据在本质上很大程度上是随机的,并且在降雨与非降雨的持续时间上高度不平衡,因此对丢失的数据恢复提出了具体的挑战。在这里,我们提出了一个两步分析,利用当前的机器学习技术,通过将任务分解为(a)雨或非雨样本的分类,以及(b)回归预测雨样本的绝对值,来估算以30分钟为间隔采样的降水数据。通过调查英国的37个气象站,这种机器学习过程比利用邻近雨量计的既定表面拟合技术更准确地预测降水数据。通过增加机器学习算法训练的可用功能,将目标地点的天气数据与外部来源的雨量计集成,提高性能,从而提供最高性能。该方法通过利用同时收集的环境数据中的信息来通知机器学习模型,以准确预测丢失的降雨数据。从弱相关变量中捕获复杂的非线性关系对于以亚小时分辨率进行数据恢复至关重要。可以开发和部署用于数据恢复的这种管道,用于以高时间分辨率对正在进行的数据集中的缺失值进行高度自动化和接近瞬时的插补。

英文摘要:

Precipitation data collected at sub-hourly resolution represents specific challenges for missing data recovery by being largely stochastic in nature and highly unbalanced in the duration of rain vs non-rain. Here we present a two-step analysis utilising current machine learning techniques for imputing precipitation data sampled at 30-minute intervals by devolving the task into (a) the classification of rain or non-rain samples, and (b) regressing the absolute values of predicted rain samples. Investigating 37 weather stations in the UK, this machine learning process produces more accurate predictions for recovering precipitation data than an established surface fitting technique utilising neighbouring rain gauges. Increasing available features for the training of machine learning algorithms increases performance with the integration of weather data at the target site with externally sourced rain gauges providing the highest performance. This method informs machine learning models by utilising information in concurrently collected environmental data to make accurate predictions of missing rain data. Capturing complex non-linear relationships from weakly correlated variables is critical for data recovery at sub-hourly resolutions. Such pipelines for data recovery can be developed and deployed for highly automated and near instantaneous imputation of missing values in ongoing datasets at high temporal resolutions.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑