Long range forecasts are the starting point of many decision support systems that need to draw inference from high-level aggregate patterns on forecasted values. State of the art time-series forecasting methods are either subject to concept drift on long-horizon forecasts, or fail to accurately predict coherent and accurate high-level aggregates.
In this work, we present a novel probabilistic forecasting method that produces forecasts that are coherent in terms of base level and predicted aggregate statistics. We achieve the coherency between predicted base-level and aggregate statistics using a novel inference method based on KL-divergence that can be solved efficiently in closed form. We show that our method improves forecast performance across both base level and unseen aggregates post inference on real datasets ranging three diverse domains. (\href{https://github.com/pratham16cse/AggForecaster}{Project URL})
With the rapid growth of renewable energy, lots of small photovoltaic (PV) prosumers emerge. Due to the uncertainty of solar power generation, there is a need for aggregated prosumers to predict solar power generation and whether solar power generation will be larger than load. This paper presents two interpretable neural networks to solve the problem: one binary classification neural network and one regression neural network. The neural networks are built using TensorFlow. The global feature importance and local feature contributions are examined by three gradient-based methods: Integrated Gr…
Faculty of Industrial Engineering, Technion—Israel Institute of Technology（以色列理工学院工业工程学院）
We consider the forecast aggregation problem in repeated settings, where the forecasts are done on a binary event. At each period multiple experts provide forecasts about an event. The goal of the aggregator is to aggregate those forecasts into a subjective accurate forecast. We assume that experts are Bayesian; namely they share a common prior, each expert is exposed to some evidence, and each expert applies Bayes rule to deduce his forecast. The aggregator is ignorant with respect to the information structure (i.e., distribution over evidence) according to which experts make their prediction…
Temporal aggregation is an intuitively appealing approach to deal with demand uncertainty. There are two types of temporal aggregation: non-overlapping and overlapping. Most of the supply chain forecasting literature has focused so far on the former and there is no research that analyses the latter for auto-correlated demands. In addition, most of the analytical research to-date assumes infinite demand series' lengths whereas, in practice, forecasting is based on finite demand histories. The length of the demand history is an important determinant of the comparative performance of the two appr…
Faculty of Engineering, University of Porto（波尔图大学工程学院）；HUUB AI Lab Porto（HUUB AI实验室波尔图）
Time series data in the retail world are particularly rich in terms of dimensionality, and these dimensions can be aggregated in groups or hierarchies. Valuable information is nested in these complex structures, which helps to predict the aggregated time series data. From a portfolio of brands under HUUB's monitoring, we selected two to explore their sales behaviour, leveraging the grouping properties of their product structure. Using statistical models, namely SARIMA, to forecast each level of the hierarchy, an optimal combination approach was used to generate more consistent forecasts in the…
University of Pavia（帕维亚大学）；Wright State University（莱特州立大学）；Wright State Research Institute（莱特州立研究所）
There are many examples of 'wisdom of the crowd' effects in which the large number of participants imparts confidence in the collective judgment of the crowd. But how do we form an aggregated judgment when the size of the crowd is limited? Whose judgments do we include, and whose do we accord the most weight? This paper considers this problem in the context of geopolitical event forecasting, where volunteer analysts are queried to give their expertise, confidence, and predictions about the outcome of an event. We develop a forecast aggregation model that integrates topical information about a …
Extending Load Forecasting from Zonal Aggregates to Individual Nodes for Transmission System Operators
School of Engineering, Stanford University（斯坦福大学工程学院）；Technical University Munich（慕尼黑技术大学）；Czech Technical University in Prague（布拉格捷克技术大学）；University of Granada（Granada大学）；Monash University（墨尔本大学）
Short-term load forecasting under EU-AI Act Requirements in Safety-Critical Environments: Results from a 41-day live challenge on the aggregated German transmission-grid load
DoFlow: Flow-based Generative Models for Interventional and Counterfactual Forecasting on Time Series
H. Milton Stewart School of Industrial and Systems Engineering（H. Milton Stewart工业与系统工程学院）；Georgia Institute of Technology（佐治亚理工学院）；Northwestern–Argonne Institute for Scientific and Engineering Excellence（西北-阿贡科学与工程卓越研究所）
Higher penetration of renewable and smart home technologies at the residential level challenges grid stability as utility-customer interactions add complexity to power system operations. In response, short-term residential load forecasting has become an increasing area of focus. However, forecasting at the residential level is challenging due to the higher uncertainties involved. Recently deep neural networks have been leveraged to address this issue. This paper investigates the capabilities of a bidirectional long short-term memory (BiLSTM) and a convolutional neural network-based BiLSTM (CNN…
Inria（法国国家信息与自动化研究所）；École Polytechnique (IPP)（巴黎综合理工学院（巴黎理工学院））；Femto-ST Institute（弗朗什-孔泰电子、力学、热学与光学科学技术研究所）；Univ. Bourg. Franche-Comté（勃艮第-弗朗什-孔泰大学）；UBFC（勃艮第-弗朗什-孔泰大学联盟）；CNRS（法国国家科学研究中心）；Orange Applications For Business（Orange商业应用公司）；Orange Labs.（奥兰治实验室）；Antonine University Hadat-Baabda（哈达特-巴卜达安东尼大学）；National University of Singapore（新加坡国立大学）
This paper investigates the problem of forecasting multivariate aggregated human mobility while preserving the privacy of the individuals concerned. Differential privacy, a state-of-the-art formal notion, has been used as the privacy guarantee in two different and independent steps when training deep learning models. On one hand, we considered \textit{gradient perturbation}, which uses the differentially private stochastic gradient descent algorithm to guarantee the privacy of each time series sample in the learning stage. On the other hand, we considered \textit{input perturbation}, which add…
Aggregated moving functional median in robust prediction of hierarchical functional time series - an application to forecasting web portal users behaviors
Cracow University of Economics（克拉科夫经济大学）；AGH University of Science and Technology（AGH科学技术大学）
University of California San Diego（加利福尼亚大学圣迭戈分校）；Microsoft Research（微软研究院）；Microsoft Gray Systems Lab（微软格雷系统实验室）；Microsoft（微软公司）；SmartApps（SmartApps公司）
Emory University（埃默里大学）；Pomona College（波莫纳学院）；Rollins School of Public Health（罗林斯公共卫生学院）
Real-time air pollution monitoring is a valuable tool for public health and environmental surveillance. In recent years, there has been a dramatic increase in air pollution forecasting and monitoring research using artificial neural networks (ANNs). Most of the prior work relied on modeling pollutant concentrations collected from ground-based monitors and meteorological data for long-term forecasting of outdoor ozone, oxides of nitrogen, and PM2.5. Given that traditional, highly sophisticated air quality monitors are expensive and are not universally available, these models cannot adequately s…
Harvard University（哈佛大学）；UC Santa Cruz（加州大学圣克鲁兹分校）
Crowdsourcing enables the solicitation of forecasts on a variety of prediction tasks from distributed groups of people. How to aggregate the solicited forecasts, which may vary in quality, into an accurate final prediction remains a challenging yet critical question. Studies have found that weighing expert forecasts more in aggregation can improve the accuracy of the aggregated prediction. However, this approach usually requires access to the historical performance data of the forecasters, which are often not available. In this paper, we study the problem of aggregating forecasts without havin…
We propose a novel approach to the problem of clustering hierarchically aggregated time-series data, which has remained an understudied problem though it has several commercial applications. We first group time series at each aggregated level, while simultaneously leveraging local and global information. The proposed method can cluster hierarchical time series (HTS) with different lengths and structures. For common two-level hierarchies, we employ a combined objective for local and global clustering over spaces of discrete probability measures, using Wasserstein distance coupled with Soft-DTW …
Tsinghua University（清华大学）；University of California, Berkeley（加州大学伯克利分校）
In time series forecasting, decomposition-based algorithms break aggregate data into meaningful components and are therefore appreciated for their particular advantages in interpretability. Recent algorithms often combine machine learning (hereafter ML) methodology with decomposition to improve prediction accuracy. However, incorporating ML is generally considered to sacrifice interpretability inevitably. In addition, existing hybrid algorithms usually rely on theoretical models with statistical assumptions and focus only on the accuracy of aggregate predictions, and thus suffer from accuracy …
In this paper, the motility model for the developed country, which United State possesses the largest economy in the world and thus serves as an ideal representation, is investigated. Early surveillance of the causes of death is critical which can allow the preparation of preventive steps against critical disease such as dengue fever. Studies reported that some search queries, especially those diseases related terms on Google Trends are essential. To this end, we include either main cause of death or the extended or the more general terminologies from Google Trends to decode the mortality rela…
CoCopula: Conformal Calibration and Copula Aggregation for Reliable Renewable Energy Forecasting
NSF Artificial Intelligence Institute for Advances in Optimization（NSF人工智能优化研究所）；H. Milton Stewart School of Industrial and Systems Engineering（H. Milton Stewart工业与系统工程学院）；Georgia Institute of Technology（佐治亚理工学院）
Asymmetric Peak-Aware Loss for Peak-Critical Time Series Forecasting
Department of Transport and Planning, Faculty of Civil Engineering and Geosciences（交通与规划系，土木与地球科学学院）；Mobility Innovation Centre Delft University of Technology（代尔夫特理工大学移动创新中心）；Delft University of Technology（代尔夫特理工大学）
A year following the initial COVID-19 outbreak in China, many countries have approved emergency vaccines. Public-health practitioners and policymakers must understand the predicted populational willingness for vaccines and implement relevant stimulation measures. This study developed a framework for predicting vaccination uptake rate based on traditional clinical data-involving an autoregressive model with autoregressive integrated moving average (ARIMA)- and innovative web search queries-involving a linear regression with ordinary least squares/least absolute shrinkage and selection operator,…
When modeling sub-national mortality rates, it is important to incorporate any possible correlation among sub-populations to improve forecast accuracy. Moreover, forecasts at the sub-national level should aggregate consistently across the forecasts at the national level. In this study, we apply a grouped multivariate functional time series to forecast Australian regional and remote age-specific mortality rates and reconcile forecasts in a group structure using various methods. Our proposed method compares favorably to a grouped univariate functional time series forecasting method by comparing …
We propose a fully probabilistic prediction model for spatially aggregated solar photovoltaic (PV) power production at an hourly time scale with lead times up to several days using weather forecasts from numerical weather prediction systems as covariates. After an appropriate logarithmic transformation of the power production, we develop a multivariate Gaussian prediction model under a Bayesian inference framework. The model incorporates the temporal error correlation yielding physically consistent forecast trajectories. Several formulations of the correlation structure are proposed and invest…
When modeling sub-national mortality rates, we should consider three features: (1) how to incorporate any possible correlation among sub-populations to potentially improve forecast accuracy through multi-population joint modeling; (2) how to reconcile sub-national mortality forecasts so that they aggregate adequately across various levels of a group structure; (3) among the forecast reconciliation methods, how to combine their forecasts to achieve improved forecast accuracy. To address these issues, we introduce an extension of grouped univariate functional time series method. We first conside…
Renewable energy sources provide a constantly increasing contribution to the total energy production worldwide. However, the power generation from these sources is highly variable due to their dependence on meteorological conditions. Accurate forecasts for the production at various temporal and spatial scales are thus needed for an efficiently operating electricity market. In this article - part 1 - we propose fully probabilistic prediction models for spatially aggregated wind power production at an hourly time scale with lead times up to several days using weather forecasts from numerical wea…
Relational AI；University of California, Merced（加州大学默塞德分校）；Carnegie Mellon University（卡内基梅隆大学）；University of Pittsburgh（匹兹堡大学）
We consider the problem of evaluating certain types of functional aggregation queries on relational data subject to additive inequalities. Such aggregation queries, with a smallish number of additive inequalities, arise naturally/commonly in many applications, particularly in learning applications. We give a relatively complete categorization of the computational complexity of such problems. We first show that the problem is NP-hard, even in the case of one additive inequality. Thus we turn to approximating the query. Our main result is an efficient algorithm for approximating, with arbitraril…
Nara Institute of Science and Technology (NAIST)（奈良先端科学技术大学院大学）
The accurate forecasting of infectious epidemic diseases such as influenza is a crucial task undertaken by medical institutions. Although numerous flu forecasting methods and models based mainly on historical flu activity data and online user-generated contents have been proposed in previous studies, no flu forecasting model targeting multiple countries using two types of data exists at present. Our paper leverages multi-task learning to tackle the challenge of building one flu forecasting model targeting multiple countries; each country as each task. Also, to develop the flu prediction model …
We present a comparative study of different probabilistic forecasting techniques on the task of predicting the electrical load of secondary substations and cabinets located in a low voltage distribution grid, as well as their aggregated power profile. The methods are evaluated using standard KPIs for deterministic and probabilistic forecasts. We also compare the ability of different hierarchical techniques in improving the bottom level forecasters' performances. Both the raw and cleaned datasets, including meteorological data, are made publicly available to provide a standard benchmark for eva…
University of Glasgow（格拉斯哥大学）；University of Warwick（华威大学）
Analysts wishing to explore multivariate data spaces, typically pose queries involving selection operators, i.e., range or radius queries, which define data subspaces of possible interest and then use aggregation functions, the results of which determine their exploratory analytics interests. However, such aggregate query (AQ) results are simple scalars and as such, convey limited information about the queried subspaces for exploratory analysis. We address this shortcoming aiding analysts to explore and understand data subspaces by contributing a novel explanation mechanism coined XAXA: eXplai…
