Statistical methods such as the Box-Jenkins method for time-series forecasting have been prominent since their development in 1970. Many researchers rely on such models as they can be efficiently estimated and also provide interpretability. However, advances in machine learning research indicate that neural networks can be powerful data modeling techniques, as they can give higher accuracy for a plethora of learning problems and datasets. In the past, they have been tried on time-series forecasting as well, but their overall results have not been significantly better than the statistical models especially for intermediate length times series data. Their modeling capacities are limited in cases where enough data may not be available to estimate the large number of parameters that these non-linear models require. This paper presents an easy to implement data augmentation method to significantly improve the performance of such networks. Our method, Augmented-Neural-Network, which involves using forecasts from statistical models, can help unlock the power of neural networks on intermediate length time-series and produces competitive results. It shows that data augmentation, when paired with Automated Machine Learning techniques such as Neural Architecture Search, can help to find the best neural architecture for a given time-series. Using the combination of these, demonstrates significant enhancement in the forecasting accuracy of three neural network-based models for a COVID-19 dataset, with a maximum improvement in forecasting accuracy by 21.41%, 24.29%, and 16.42%, respectively, over the neural networks that do not use augmented data.
University of Toronto（多伦多大学）；University of Wisconsin（威斯康星大学）；IIT Bombay（印度理工学院孟买分校）；Carnegie Mellon University（卡内基梅隆大学）；Stanford University（斯坦福大学）
The challenge that climate change poses to humanity has spurred a rapidly developing field of artificial intelligence research focused on climate change applications. The climate change AI (CCAI) community works on a diverse, challenging set of problems which often involve physics-constrained ML or heterogeneous spatiotemporal data. It would be desirable to use automated machine learning (AutoML) techniques to automatically find high-performing architectures and hyperparameters for a given dataset. In this work, we benchmark popular AutoML libraries on three high-leverage CCAI applications: cl…
The Chinese University of Hong Kong（香港中文大学）；International Digital Economy Academy（国际数字经济研究院）
Data augmentation (DA) has become a de facto solution to expand training data size for deep learning. With the proliferation of deep models for time series analysis, various time series DA techniques are proposed in the literature, e.g., cropping-, warping-, flipping-, and mixup-based methods. However, these augmentation methods mainly apply to time series classification and anomaly detection tasks. In time series forecasting (TSF), we need to model the fine-grained temporal relationship within time series segments to generate accurate forecasting results given data in a look-back window. Exis…
University of California, San Diego（加利福尼亚大学圣迭戈分校）；Amazon（亚马逊）
Time-series data augmentation mitigates the issue of insufficient training data for deep learning models. Yet, existing augmentation methods are mainly designed for classification, where class labels can be preserved even if augmentation alters the temporal dynamics. We note that augmentation designed for forecasting requires diversity as well as coherence with the original temporal dynamics. As time-series data generated by real-life physical processes exhibit characteristics in both the time and frequency domains, we propose to combine Spectral and Time Augmentation (STAug) for generating mo…
In this work, we propose a deep learning approach to forecasting state-level COVID-19 trends of weekly cumulative death in the United States (US) and incident cases in Germany. This approach includes a transformer model, an ensemble method, and a data augmentation technique for time series. We arrange the inputs of the transformer in such a way that predictions for different states can attend to the trends of the others. To overcome the issue of scarcity of training data for this COVID-19 pandemic, we have developed a novel data augmentation technique to generate useful data for training. More…
Sliding-Window Reordering with Overlap Averaging: A Simple Time-Domain Augmentation for Multivariate Forecasting
Data Science Group, University of Hildesheim（希尔德斯海姆大学数据科学组）；Information Systems and Machine Learning Lab (ISMLL), University of Hildesheim（希尔德斯海姆大学信息系统与机器学习实验室）
Forecasting models that are trained across sets of many time series, known as Global Forecasting Models (GFM), have shown recently promising results in forecasting competitions and real-world applications, outperforming many state-of-the-art univariate forecasting techniques. In most cases, GFMs are implemented using deep neural networks, and in particular Recurrent Neural Networks (RNN), which require a sufficient amount of time series to estimate their numerous model parameters. However, many time series databases have only a limited number of time series. In this study, we propose a novel, …
Online Data Augmentation for Forecasting with Deep Learning
Faculdade de Engenharia da Universidade do Porto（波尔图大学工程学院）；Laboratory for Artificial Intelligence and Computer Science (LIACC)（人工智能与计算机科学实验室）；INESC TEC；Fraunhofer Portugal AICOS（弗劳恩霍夫葡萄牙AICOS）
Multi-horizon probabilistic time series forecasting has wide applicability to real-world tasks such as demand forecasting. Recent work in neural time-series forecasting mainly focus on the use of Seq2Seq architectures. For example, MQTransformer - an improvement of MQCNN - has shown the state-of-the-art performance in probabilistic demand forecasting. In this paper, we consider incorporating cross-entity information to enhance model performance by adding a cross-entity attention mechanism along with a retrieval mechanism to select which entities to attend over. We demonstrate how our new neura…
Embedding-Space Data Augmentation to Prevent Membership Inference Attacks in Clinical Time Series Forecasting
Department of Computational Linguistics Heidelberg University Germany（计算语言学系海德堡大学）；Department of Computational Linguistics & Interdisciplinary Center for Scientific Computing (IWR) Heidelberg University Germany（计算语言学系与跨学科科学计算中心（IWR）海德堡大学）
Leibniz University Hannover（汉诺威莱布尼茨大学）；Ludwig-Maximilian University（路德维希-马克西米利安大学）；Fraunhofer Institute for Integrated Circuits (IIS)（弗劳恩霍夫集成电路研究所）；University of Freiburg（弗莱堡大学）；Bosch Center for Artificial Intelligence（博世人工智能中心）
Recent years have witnessed tremendously improved efficiency of Automated Machine Learning (AutoML), especially Automated Deep Learning (AutoDL) systems, but recent work focuses on tabular, image, or NLP tasks. So far, little attention has been paid to general AutoDL frameworks for time series forecasting, despite the enormous success in applying different novel architectures to such tasks. In this paper, we propose an efficient approach for the joint optimization of neural architecture and hyperparameters of the entire data processing pipeline for time series forecasting. In contrast to commo…
Meta-learning and Data Augmentation for Stress Testing Forecasting Models
Faculdade de Engenharia da Universidade do Porto（波尔图大学工程学院）；Laboratory for Artificial Intelligence and Computer Science (LIACC)（人工智能与计算机科学实验室（LIACC））；Fraunhofer Portugal AICOS（弗劳恩霍夫葡萄牙AICOS研究所）
Karlsruhe Institute of Technology（卡尔斯鲁厄理工学院）；RWTH Aachen University（亚琛工业大学）；Forschungszentrum Jülich GmbH（于利希研究中心有限公司）
Time series forecasting is fundamental for various use cases in different domains such as energy systems and economics. Creating a forecasting model for a specific use case requires an iterative and complex design process. The typical design process includes the five sections (1) data pre-processing, (2) feature engineering, (3) hyperparameter optimization, (4) forecasting method selection, and (5) forecast ensembling, which are commonly organized in a pipeline structure. One promising approach to handle the ever-growing demand for time series forecasts is automating this design process. The p…
We consider two-state Non-Homogeneous Hidden Markov Models (NHHMMs) for forecasting univariate time series. Given a set of predictors, the time series are modeled via predictive regressions with state dependent coefficients and time-varying transition probabilities that depend on the predictors via a logistic function. In a hidden Markov setting, inference for logistic regression coefficients becomes complicated and in some cases impossible due to convergence issues. In this paper, we aim to address this problem using a new latent variable scheme that utilizes the Pólya-Gamma class of distribu…
Simula Research Laboratory（Simula研究实验室）；University of Thessaly（色萨利大学）；University of Oslo（奥斯陆大学）；Simula Metropolitan（Simula大都会中心）
Bandwidth forecasting in Mobile Broadband (MBB) networks is a challenging task, particularly when coupled with a degree of mobility. In this work, we introduce HINDSIGHT++, an open-source R-based framework for bandwidth forecasting experimentation in MBB networks with Long Short Term Memory (LSTM) networks. We instrument HINDSIGHT++ following an Automated Machine Learning (AutoML) paradigm to first, alleviate the burden of data preprocessing, and second, enhance performance related aspects. We primarily focus on bandwidth forecasting for Fifth Generation (5G) networks. In particular, we levera…
Enhancing Prediction and Analysis of UK Road Traffic Accident Severity Using AI: Integration of Machine Learning, Econometric Techniques, and Time Series Forecasting in Public Health Research
L-GTA: Latent Generative Modeling for Time Series Augmentation
Faculdade de Engenharia da Universidade do Porto（葡萄牙波尔图大学工程学院）；Fraunhofer Portugal AICOS（德国弗劳恩霍夫葡萄牙AICOS研究所）；University of Coimbra（科英布拉大学）；Dalhousie University（达尔豪斯大学）
A novel forecasting framework combining virtual samples and enhanced Transformer models for tourism demand forecasting
School of Computer, Electronics and Information, Guangxi University（广西大学计算机与电子信息学院）；School of Electrical Engineering, Guangxi University（广西大学电气工程学院）；School of Mathematics and Statistics, Xuzhou University of Technology（徐州工程学院数学与统计学院）
Aimsun SLU（Aimsun SLU公司）；DeustoTech（德乌斯托技术中心）；Faculty of Engineering, University of Deusto（德乌斯托大学工程学院）；University of Deusto（德乌斯托大学）；IKERBASQUE, Basque Foundation for Science（巴斯克科学基金会IKERBASQUE）；University of Granada（格拉纳达大学）
Intelligent Transportation Systems are producing tons of hardly manageable traffic data, which motivates the use of Machine Learning (ML) for data-driven applications, such as Traffic Forecasting (TF). TF is gaining relevance due to its ability to mitigate traffic congestion by forecasting future traffic states. However, TF poses one big challenge to the ML paradigm, known as the Model Selection Problem (MSP): deciding the most suitable combination of data preprocessing techniques and ML method for traffic data collected under different transportation circumstances. In this context, Automated …
TSGym: Design Choices for Deep Multivariate Time-Series Forecasting
AI Lab, Shanghai University of Finance and Economics（上海财经大学金融学院人工智能实验室）；Ant Group（蚂蚁集团）；MoE Key Laboratory of Interdisciplinary Research of Computation and Economics（计算与经济学交叉学科关键实验室）
Time Series Data Augmentation as an Imbalanced Learning Problem
Faculdade de Engenharia da Universidade do Porto（波尔图大学工程学院）；Laboratory for Artificial Intelligence and Computer Science (LIACC)（人工智能与计算机科学实验室（LIACC））；Notre Dame University（圣母大学）；Fraunhofer Portugal AICOS（弗劳恩霍夫葡萄牙AICOS机构）
NXAI GmbH（NXAI公司）；ELLIS Unit, LIT AI Lab, Institute for Machine Learning, JKU Linz（ELLIS单位、LIT AI实验室、机器学习研究所）；Interdisciplinary Transformation University Austria（跨学科转型大学奥地利）
Recent research on time series foundation models has primarily focused on forecasting, leaving it unclear how generalizable their learned representations are. In this study, we examine whether frozen pre-trained forecasting models can provide effective representations for classification. To this end, we compare different representation extraction strategies and introduce two model-agnostic embedding augmentations. Our experiments show that the best forecasting models achieve classification accuracy that matches or even surpasses that of state-of-the-art models pre-trained specifically for clas…
Time series forecasting is an important yet challenging task. Though deep learning methods have recently been developed to give superior forecasting results, it is crucial to improve the interpretability of time series models. Previous interpretation methods, including the methods for general neural networks and attention-based methods, mainly consider the interpretation in the feature dimension while ignoring the crucial temporal dimension. In this paper, we present the series saliency framework for temporal interpretation for multivariate time series forecasting, which considers the forecast…
SunPower Corporation（SunPower公司）；SolarEdge Technologies（SolarEdge科技公司）；Case Western Reserve University（凯斯西储大学）；C2 Energy Capital（C2能源资本公司）；Sandia National Labs（桑迪亚国家实验室）
The integration of the global Photovoltaic (PV) market with real time data-loggers has enabled large scale PV data analytical pipelines for power forecasting and long-term reliability assessment of PV fleets. Nevertheless, the performance of PV data analysis heavily depends on the quality of PV timeseries data. This paper proposes a novel Spatio-Temporal Denoising Graph Autoencoder (STD-GAE) framework to impute missing PV Power Data. STD-GAE exploits temporal correlation, spatial coherence, and value dependencies from domain knowledge to recover missing data. Experimental results show that STD…
An AutoML Framework using AutoGluonTS for Forecasting Seasonal Extreme Temperatures
Facultad de Ingeniería, Universidad de la República（拉普拉塔大学工程学院）；Departamento de Física de la Tierra y Astrofísica, Universidad Complutense de Madrid（马德里complutense大学地球与天体物理系）；Faculty of Exact and Natural Sciences, Universidad de Buenos Aires (UBA)（布宜诺斯艾利斯大学（UBA）精确与自然科学学院）；Technical University of Denmark（丹麦技术大学）；INRIA（法国国家信息与自动化技术研究所）
Addressing Concept Shift in Online Time Series Forecasting: Detect-then-Adapt
Chinese Academy of Sciences (CASIA)（中国科学院）；University of Chinese Academy of Sciences (UCAS)（中国科学院大学）；Alibaba Group（阿里巴巴集团）；Squirrel AI Group（松鼠AI智适应教育集团）；Meta AI Group（元宇宙人工智能集团）
Grasynda: Graph-based Synthetic Time Series Generation
Department of Informatics, University of Minho, Braga, Portugal（明德大学信息学院）；Faculdade de Engenharia da Universidade do Porto, Porto, Portugal（波尔图大学工程学院）；Laboratory for Artificial Intelligence and Computer Science (LIACC), Portugal（人工智能与计算机科学实验室）；Fraunhofer Portugal AICOS, Portugal（弗劳恩霍夫葡萄牙人工智能与计算科学中心）；HASLab - INESCTEC（HAS实验室 - INESCTEC）
Compositionality in Time Series: A Proof of Concept using Symbolic Dynamics and Compositional Data Augmentation
Department of Computational Linguistics（计算语言学系）；Heidelberg University（海德堡大学）；Department of Computational Linguistics and Interdisciplinary Center for Scientific Computing (IWR)（计算语言学系和科学计算跨学科中心）
