本文提出基于 von Mises 分布的循环回归树与随机森林，用于循环响应变量的分布建模和概率预测，并在机场风向预测中验证其效果。
摘要：
尽管循环数据出现在广泛的科学领域中，但用于循环响应变量的分布建模和概率预测的方法仍然相当有限。现有方法大多建立在广义线性模型和广义加性模型框架之上，这些方法通常难以优化和解释。因此，我们提出循环回归树和随机森林，作为一种直观且相对容易拟合的替代方法。在以往关于建模循环均值的树方法思想基础上，我们针对树和森林提出了一种分布方法，可基于 von Mises 分布生成概率预测。所得基于树的模型通过利用可用协变量将数据划分为足够同质的子组来简化估计过程，从而可以在每个子组中对循环响应拟合一个不含进一步协变量的简单 von Mises 分布。这些循环回归树易于解释，能够捕捉非线性效应和交互作用，并自动选择与 von Mises 分布的位置和/或尺度变化相关的协变量。将循环回归树集合组合成循环回归森林，可得到 von Mises 分布的局部自适应似然估计量，该估计量能够对协变量效应进行正则化和平滑。新方法在奥地利两个机场的概率风向预测案例研究中进行了评估，并将其他常用方法作为基准。
英文摘要：
While circular data occur in a wide range of scientific fields, the methodology for distributional modeling and probabilistic forecasting of circular response variables is rather limited. Most of the existing methods are built on the framework of generalized linear and additive models, which are often challenging to optimize and to interpret. Therefore, we suggest circular regression trees and random forests as an intuitive alternative approach that is relatively easy to fit. Building on previous ideas for trees modeling circular means, we suggest a distributional approach for both trees and forests yielding probabilistic forecasts based on the von Mises distribution. The resulting tree-based models simplify the estimation process by using the available covariates for partitioning the data into sufficiently homogeneous subgroups so that a simple von Mises distribution without further covariates can be fitted to the circular response in each subgroup. These circular regression trees are straightforward to interpret, can capture nonlinear effects and interactions, and automatically select the relevant covariates that are associated with either location and/or scale changes in the von Mises distribution. Combining an ensemble of circular regression trees to a circular regression forest yields a local adaptive likelihood estimator for the von Mises distribution that can regularize and smooth the covariate effects. The new methods are evaluated in a case study on probabilistic wind direction forecasting at two Austrian airports, considering other common approaches as a benchmark.
Circular data arise in many areas of application. Recently, there has been interest in looking at circular data collected separately over time and over space. Here, we extend some of this work to the spatio-temporal setting, introducing space-time dependence. We accommodate covariates, implement full kriging and forecasting, and also allow for a nugget which can be time dependent. We work within a Bayesian framework, introducing suitable latent variables to facilitate Markov chain Monte Carlo (MCMC) model fitting. The Bayesian framework enables us to implement full inference, obtaining predict…
Threshold Autoregressive (TAR) models have been widely used by statisticians for non-linear time series forecasting during the past few decades, due to their simplicity and mathematical properties. On the other hand, in the forecasting community, general-purpose tree-based regression algorithms (forests, gradient-boosting) have become popular recently due to their ease of use and accuracy. In this paper, we explore the close connections between TAR models and regression trees. These enable us to use the rich methodology from the literature on TAR models to define a hierarchical TAR model as a …
Kaunas University of Technology（考纳斯理工大学）；Rivile（Rivile私人有限责任公司）
Amounts of historical data collected increase and business intelligence applicability with automatic forecasting of time series are in high demand. While no single time series modeling method is universal to all types of dynamics, forecasting using an ensemble of several methods is often seen as a compromise. Instead of fixing ensemble diversity and size, we propose to predict these aspects adaptively using meta-learning. Meta-learning here considers two separate random forest regression models, built on 390 time-series features, to rank 22 univariate forecasting methods and recommend ensemble…
Forecasting and Explaining the Phillips Curve: A SHAP-Based Comparison of Machine Learning and Traditional Time-Series Models for Canadian Unemployment and Inflation
Department of Economics, University of Ottawa（渥太华大学经济学系）
University of Liège（列日大学）；Polytechnique Montréal（蒙特利尔理工学院）
In this paper, we present machine learning approaches for characterizing and forecasting the short-term demand for on-demand ride-hailing services. We propose the spatio-temporal estimation of the demand that is a function of variable effects related to traffic, pricing and weather conditions. With respect to the methodology, a single decision tree, bootstrap-aggregated (bagged) decision trees, random forest, boosted decision trees, and artificial neural network for regression have been adapted and systematically compared using various statistics, e.g. R-square, Root Mean Square Error (RMSE), …
Institute for Development and Research in Banking Technology（银行技术发展与研究所）；University of Hyderabad（海得拉巴大学）
This paper proposes to model chaos in the ATM cash withdrawal time series of a big Indian bank and forecast the withdrawals using deep learning methods. It also considers the importance of day-of-the-week and includes it as a dummy exogenous variable. We first modelled the chaos present in the withdrawal time series by reconstructing the state space of each series using the lag, and embedding dimension found using an auto-correlation function and Cao's method. This process converts the uni-variate time series into multi variate time series. The "day-of-the-week" is converted into seven feature…
To obtain a probabilistic model for a dependent variable based on some set of explanatory variables, a distributional approach is often adopted where the parameters of the distribution are linked to regressors. In many classical models this only captures the location of the distribution but over the last decade there has been increasing interest in distributional regression approaches modeling all parameters including location, scale, and shape. Notably, so-called non-homogeneous Gaussian regression (NGR) models both mean and variance of a Gaussian response and is particularly popular in weath…
São Paulo School of Economics, Getulio Vargas Foundation（圣保罗经济学院，杰图利奥·瓦加斯基金会）；Pontifical Catholic University of Rio de Janeiro（里约热内卢天主教大学）；Getulio Vargas Foundation（杰图利奥·瓦加斯基金会）
In this paper we survey the most recent advances in supervised machine learning and high-dimensional models for time series forecasting. We consider both linear and nonlinear alternatives. Among the linear methods we pay special attention to penalized regressions and ensemble of models. The nonlinear methods considered in the paper include shallow and deep neural networks, in their feed-forward and recurrent versions, and tree-based methods, such as random forests and boosted trees. We also consider ensemble and hybrid models by combining ingredients from different alternatives. Tests for supe…
Forecasting the future development in quality and value of professional football players
1 Delft Institute of Applied Mathematics, Delft University of Technology, 2628 CD Delft, The Netherlands；\ 2 Data Science \& Computer Vision, SciSports, 3703 HX Zeist, The Netherlands
A year following the initial COVID-19 outbreak in China, many countries have approved emergency vaccines. Public-health practitioners and policymakers must understand the predicted populational willingness for vaccines and implement relevant stimulation measures. This study developed a framework for predicting vaccination uptake rate based on traditional clinical data-involving an autoregressive model with autoregressive integrated moving average (ARIMA)- and innovative web search queries-involving a linear regression with ordinary least squares/least absolute shrinkage and selection operator,…
Enhancing Prediction and Analysis of UK Road Traffic Accident Severity Using AI: Integration of Machine Learning, Econometric Techniques, and Time Series Forecasting in Public Health Research
Winds from the North-West quadrant and lack of precipitation are known to lead to an increase of PM10 concentrations over a residential neighborhood in the city of Taranto (Italy). In 2012 the local government prescribed a reduction of industrial emissions by 10% every time such meteorological conditions are forecasted 72 hours in advance. Wind forecasting is addressed using the Weather Research and Forecasting (WRF) atmospheric simulation system by the Regional Environmental Protection Agency. In the context of distributions-oriented forecast verification, we propose a comprehensive model-bas…
Machine Learning and ARIMA Model Averaging for Adaptive Public Health Forecasting: Comparative Evaluation and an Ontario COVID-19 Case Study
Public Health Ontario（安大略省公共卫生局）；Dalla Lana School of Public Health, University of Toronto（多伦多大学达拉·拉纳公共卫生学院）；University of Toronto（多伦多大学）；York University（约克大学）
Time-dependent data collected in studies of Alzheimer's disease usually has missing and irregularly sampled data points. For this reason time series methods which assume regular sampling cannot be applied directly to the data without a pre-processing step. In this paper we use a machine learning method to learn the relationship between pairs of data points at different time separations. The input vector comprises a summary of the time series history and includes both demographic and non-time varying variables such as genetic data. The dataset used is from the 2017 TADPOLE grand challenge which…
University of Patras（帕特雷大学）；Roma Tre University（罗马第三大学）；National Technical University of Athens（雅典国立技术大学）
Machine and statistical learning algorithms can be reliably automated and applied at scale. Therefore, they can constitute a considerable asset for designing practical forecasting systems, such as those related to urban water demand. Quantile regression algorithms are statistical and machine learning algorithms that can provide probabilistic forecasts in a straightforward way, and have not been applied so far for urban water demand forecasting. In this work, we aim to fill this gap by automating and extensively comparing several quantile-regression-based practical systems for probabilistic one…
Los Alamos National Laboratory（洛斯阿拉莫斯国家实验室）；University of Pittsburgh（匹兹堡大学）
Standard supervised learning procedures are validated against a test set that is assumed to have come from the same distribution as the training data. However, in many problems, the test data may have come from a different distribution. We consider the case of having many labeled observations from one distribution, $P_1$, and making predictions at unlabeled points that come from $P_2$. We combine the high predictive accuracy of random forests (Breiman, 2001) with an importance sampling scheme, where the splits and predictions of the base-trees are done in a weighted manner, which we call Local…
