A scoring rule is a function of a probabilistic forecast and a corresponding outcome that is used to evaluate forecast performance. A wide range of scoring rules have been defined over time and there is some debate as to which are the most appropriate for evaluating the performance of forecasts of sporting events. This paper focuses on forecasts of the outcomes of football matches. The ranked probability score (RPS) is often recommended since it is `sensitive to distance', that is it takes into account the ordering in the outcomes (a home win is `closer' to a draw than it is to an away win, for example). In this paper, this reasoning is disputed on the basis that it adds nothing in terms of the actual aims of using scoring rules. A related property of scoring rules is locality. A scoring rule is local if it only takes the probability placed on the outcome into consideration. Two simulation experiments are carried out in the context of football matches to compare the performance of the RPS, which is non-local and sensitive to distance, the Brier score, which is non-local and insensitive to distance, and the ignorance score, which is local and insensitive to distance. The ignorance score is found to outperform both the RPS and the Brier score, casting doubt on the value of non-locality and sensitivity to distance as properties of scoring rules in this context.
This paper considers the use of observed and predicted match statistics as inputs to forecasts of the outcomes of football matches. It is shown that, were it possible to know the match statistics in advance, highly informative forecasts of the match outcome could be made. Whilst, in practice, match statistics are never available prior to the match, this leads to a simple philosophy. If match statistics can be predicted pre-match, and if those predictions are accurate enough, it follows that informative match forecasts can be made. It is shown that Generalised Attacking Performance (GAP) rating…
Huge amounts of money are invested every year by football clubs on transfers. For both growth and survival, it is crucial for recruiting departments to make smart choices when targeting players. Therefore, it is very important to identify the right parameters to monitor to predict market value. The following paper aims at determining the relevant features that successfully forecast future value for football players. Success is measured against their market value from TransferMarkt. To select prominent features, we use Lasso regressions and Random Forest algorithms. Some obvious variables are s…
Football forecasting models traditionally rate teams on past match results, that is based on the number of goals scored. Goals, however, involve a high element of chance and thus past results often do not reflect the performances of the teams. In recent years, it has become increasingly clear that accounting for other match events such as shots at goal can provide a better indication of the relative strengths of two teams than the number of goals scored. Forecast models based on this information have been shown to be successful in outperforming those based purely on match results. A notable we…
This report is devoted to the forecast of the UEFA EURO 2020, Europe's continental football championship, taking place across Europe in June/July 2021. We present the simulation results for this tournament, where the simulations are based on a zero-inflated generalized Poisson regression model that includes the Elo points of the participating teams and the location of the matches as covariates and incorporates differences of team-specific skills. The proposed model allows predictions in terms of probabilities in order to quantify the chances for each team to reach a certain stage of the tourna…
The FIFA Men's World Cup Tournament (WCT) is the most important football (soccer) competition, attracting worldwide attention. A popular practice among football fans in Brazil is to organize contests in which each participant informs guesses on the final score of each match. The participants are then ranked according to some scoring rule. Inspired by these contests, we created a website to hold an online contest, in which participants were asked for their probabilities on the outcomes of upcoming matches of the WCT. After each round of the tournament, the ranking of all users based on a proper…
Match forecasts in UEFA club competitions: Elo ratings versus Transfermarkt valuations
Institute of Economics, ELTE（匈牙利罗兰大学经济研究所）；Institute for Computer Science and Control (SZTAKI), Hungarian Research Network (HUN-REN)（匈牙利研究网络计算机科学与控制研究所）；Hungarian University of Sports Science（匈牙利体育科学大学）；Corvinus University of Budapest (BCE)（布达佩斯考文纽斯大学）
本文比较了 Elo 评分与 Transfermarkt 估值在预测欧足联俱乐部赛事比赛结果上的表现，发现两者单独使用时准确性相近，结合使用略有提升，且基于 Elo 评分的种子排位接近最优。
Verifying probabilistic forecasts for extreme events is a highly active research area because popular media and public opinions are naturally focused on extreme events, and biased conclusions are readily made. In this context, classical verification methods tailored for extreme events, such as thresholded and weighted scoring rules, have undesirable properties that cannot be mitigated, and the well-known continuous ranked probability score (CRPS) is no exception. In this paper, we define a formal framework for assessing the behavior of forecast evaluation procedures with respect to extreme eve…
This article is devoted to the forecast of the Africa Cup of Nations 2019 football tournament. It is based on a Poisson regression model that includes the Elo points of the participating teams as covariates and incorporates differences of team-specific skills. The proposed model allows predictions in terms of probabilities in order to quantify the chances for each team to reach a certain stage of the tournament. Monte Carlo simulations are used to estimate the outcome of each single match of the tournament and hence to simulate the whole tournament itself. The model is fitted on all football g…
For practical reasons, many forecasts of case, hospitalization and death counts in the context of the current COVID-19 pandemic are issued in the form of central predictive intervals at various levels. This is also the case for the forecasts collected in the COVID-19 Forecast Hub (https://covid19forecasthub.org/). Forecast evaluation metrics like the logarithmic score, which has been applied in several infectious disease forecasting challenges, are then not available as they require full predictive distributions. This article provides an overview of how established methods for the evaluation o…
TU Dortmund University（多特蒙德工业大学）；Molde University College（莫尔德大学学院）；Ghent University（根特大学）；Technical University of Munich（慕尼黑工业大学）；Universität Innsbruck（因斯布鲁克大学）
Three state-of-the-art statistical ranking methods for forecasting football matches are combined with several other predictors in a hybrid machine learning model. Namely an ability estimate for every team based on historic matches; an ability estimate for every team based on bookmaker consensus; average plus-minus player ratings based on their individual performances in their home clubs and national teams; and further team covariates (e.g., market value, team structure) and country-specific socio-economic factors (population, GDP). The proposed combined approach is used for learning the number…
Royal Netherlands Meteorological Institute (KNMI)（荷兰皇家气象研究所（KNMI））；Utrecht University（乌得勒支大学）
Current statistical post-processing methods for probabilistic weather forecasting are not capable of using full spatial patterns from the numerical weather prediction (NWP) model. In this paper we incorporate spatial wind speed information by using convolutional neural networks (CNNs) and obtain probabilistic wind speed forecasts in the Netherlands for 48 hours ahead, based on KNMI's deterministic Harmonie-Arome NWP model. The probabilistic forecasts from the CNNs are shown to have higher Brier skill scores for medium to higher wind speeds, as well as a better continuous ranked probability sco…
In the field of numerical weather prediction (NWP), the probabilistic distribution of the future state of the atmosphere is sampled with Monte-Carlo-like simulations, called ensembles. These ensembles have deficiencies (such as conditional biases) that can be corrected thanks to statistical post-processing methods. Several ensembles exist and may be corrected with different statistiscal methods. A further step is to combine these raw or post-processed ensembles. The theory of prediction with expert advice allows us to build combination algorithms with theoretical guarantees on the forecast per…
From electricity prices to profits: multidimensional probabilistic forecasting for BESS trading
organization= Department of Applied Mathematics, Wroc aw University of Science；organization= Department of Operations Research；Business Intelligence, Wroc aw University of Science
The increased usage of solar energy places additional importance on forecasts of solar radiation. Solar panel power production is primarily driven by the amount of solar radiation and it is therefore important to have accurate forecasts of solar radiation. Accurate forecasts that also give information on the forecast uncertainties can help users of solar energy to make better solar radiation based decisions related to the stability of the electrical grid. To achieve this, we apply statistical post-processing techniques that determine relationships between observations of global radiation (made…
Achieving Skilled and Reliable Daily Probabilistic Forecasts of Wind Power at Subseasonal-to-Seasonal Timescales over France
Atos Inno’Lab TS Bezons（阿托斯创新实验室TS贝宗）；EDF R&D Lab（EDF研发实验室）；Laboratoire des Sciences du Climat & de l’Environnement（气候与环境科学实验室）；CEA（法国国家科学研究中心）；CNRS（法国国家科学研究中心）；UVSQ（法国大学巴黎-萨克雷）；Université Paris-Saclay（巴黎-萨克雷大学）；Laboratoire de Mathématiques d’Orsay（奥赛数学实验室）
Machine Learning and ARIMA Model Averaging for Adaptive Public Health Forecasting: Comparative Evaluation and an Ontario COVID-19 Case Study
Public Health Ontario（安大略省公共卫生局）；Dalla Lana School of Public Health, University of Toronto（多伦多大学达拉·拉纳公共卫生学院）；University of Toronto（多伦多大学）；York University（约克大学）
