Data augmentation with generative adversarial networks (GANs) has been popular for class imbalance problems, mainly for pattern classification and computer vision-related applications. Extreme value forecasting is a challenging field that has various applications from finance to climate change problems. In this study, we present a data augmentation framework for extreme value forecasting. In this framework, our focus is on forecasting extreme values using deep learning models in combination with data augmentation models such as GANs and synthetic minority oversampling technique (SMOTE). We use deep learning models such as convolutional long short-term memory (Conv-LSTM) and bidirectional long short-term memory (BD-LSTM) networks for multistep ahead prediction featuring extremes. We investigate which data augmentation models are the most suitable, taking into account the prediction accuracy overall and at extreme regions, along with computational efficiency. We also present novel strategies for incorporating data augmentation, considering extreme values based on a relevance function. Our results indicate that the SMOTE-based strategy consistently demonstrated superior adaptability, leading to improved performance across both short- and long-horizon forecasts. Conv-LSTM and BD-LSTM exhibit complementary strengths: the former excels in periodic, stable datasets, while the latter performs better in chaotic or non-stationary sequences.
Data augmentation through multivariate scenario forecasting in Data Centers using Generative Adversarial Networks
IIT - Instituto de Investigación Tecnológica, Universidad Pontificia Comillas（科米利亚斯主教大学技术研究所（IIT））；Universidad Politécnica de Madrid（马德里理工大学）；CCS - Center for Computational Simulation（计算模拟中心（CCS））
National University of Singapore（新加坡国立大学）；University of Hong Kong（香港大学）；JD Intelligent Cities Research（京东智能城市研究院）；JD iCity, JD Technology（京东科技京东智联云）
Deep learning models are modern tools for spatio-temporal graph (STG) forecasting. Though successful, we argue that data scarcity is a key factor limiting their recent improvements. Meanwhile, contrastive learning has been an effective method for providing self-supervision signals and addressing data scarcity in various domains. In view of this, one may ask: can we leverage the additional signals from contrastive learning to alleviate data scarcity, so as to benefit STG forecasting? To answer this question, we present the first systematic exploration on incorporating contrastive learning into …
STS-CCL: Spatial-Temporal Synchronous Contextual Contrastive Learning for Urban Traffic Forecasting
Zhejiang University（浙江大学）；South China University of Technology（华南理工大学）；The University of Sydney（悉尼大学）；Chongqing University（重庆大学）；Zhejiang Institute of Industry and Information Technology（浙江工业信息技术研究院）
The Chinese University of Hong Kong（香港中文大学）；International Digital Economy Academy（国际数字经济研究院）
Data augmentation (DA) has become a de facto solution to expand training data size for deep learning. With the proliferation of deep models for time series analysis, various time series DA techniques are proposed in the literature, e.g., cropping-, warping-, flipping-, and mixup-based methods. However, these augmentation methods mainly apply to time series classification and anomaly detection tasks. In time series forecasting (TSF), we need to model the fine-grained temporal relationship within time series segments to generate accurate forecasting results given data in a look-back window. Exis…
University of California, San Diego（加利福尼亚大学圣迭戈分校）；Amazon（亚马逊）
Time-series data augmentation mitigates the issue of insufficient training data for deep learning models. Yet, existing augmentation methods are mainly designed for classification, where class labels can be preserved even if augmentation alters the temporal dynamics. We note that augmentation designed for forecasting requires diversity as well as coherence with the original temporal dynamics. As time-series data generated by real-life physical processes exhibit characteristics in both the time and frequency domains, we propose to combine Spectral and Time Augmentation (STAug) for generating mo…
A novel forecasting framework combining virtual samples and enhanced Transformer models for tourism demand forecasting
School of Computer, Electronics and Information, Guangxi University（广西大学计算机与电子信息学院）；School of Electrical Engineering, Guangxi University（广西大学电气工程学院）；School of Mathematics and Statistics, Xuzhou University of Technology（徐州工程学院数学与统计学院）
APPCAIR；BITS Pilani, K K Birla Goa Campus（皮拉尼比尔拉理工学院果阿校区）；TCS Research（塔塔咨询服务公司研究院）
Deep-learning techniques have been successfully used for time-series forecasting and have often shown superior performance on many standard benchmark datasets as compared to traditional techniques. Here we present a comprehensive and comparative study of performance of deep-learning techniques for forecasting prices in financial markets. We benchmark state-of-the-art deep-learning baselines, such as NBeats, etc., on data from currency as well as stock markets. We also generate synthetic data using a fuzzy-logic based model of demand driven by technical rules such as moving averages, which are …
Multi-horizon probabilistic time series forecasting has wide applicability to real-world tasks such as demand forecasting. Recent work in neural time-series forecasting mainly focus on the use of Seq2Seq architectures. For example, MQTransformer - an improvement of MQCNN - has shown the state-of-the-art performance in probabilistic demand forecasting. In this paper, we consider incorporating cross-entity information to enhance model performance by adding a cross-entity attention mechanism along with a retrieval mechanism to select which entities to attend over. We demonstrate how our new neura…
Improving age prediction: Utilizing LSTM-based dynamic forecasting for data augmentation in multivariate time series analysis
Tri-Institutional Center for Translational Research in Neuroimaging and Data Science (TReNDS): Georgia State University, Georgia Institute of Technology, Emory University（转化神经影像与数据科学三机构中心（TReNDS）：佐治亚州立大学、佐治亚理工学院、埃默里大学）
University of California San Diego（加州大学圣地亚哥分校）；Northeastern University（东北大学）
Exploiting symmetry in dynamical systems is a powerful way to improve the generalization of deep learning. The model learns to be invariant to transformation and hence is more robust to distribution shift. Data augmentation and equivariant networks are two major approaches to injecting symmetry into learning. However, their exact role in improving generalization is not well understood. In this work, we derive the generalization bounds for data augmentation and equivariant networks, characterizing their effect on learning in a unified framework. Unlike most prior theories for the i.i.d. setting…
Forecasting models that are trained across sets of many time series, known as Global Forecasting Models (GFM), have shown recently promising results in forecasting competitions and real-world applications, outperforming many state-of-the-art univariate forecasting techniques. In most cases, GFMs are implemented using deep neural networks, and in particular Recurrent Neural Networks (RNN), which require a sufficient amount of time series to estimate their numerous model parameters. However, many time series databases have only a limited number of time series. In this study, we propose a novel, …
Forecasting-based Biomedical Time-series Data Synthesis for Open Data and Robust AI
School of Electrical Engineering, KAIST, Daejeon, Republic of Korea（韩国成均馆大学电气工程学院）；Department of Applied AI, Hansung University, Seoul, Republic of Korea（汉城市立大学应用人工智能系）
One of the limiting factors in training data-driven, rare-event prediction algorithms is the scarcity of the events of interest resulting in an extreme imbalance in the data. There have been many methods introduced in the literature for overcoming this issue; simple data manipulation through undersampling and oversampling, utilizing cost-sensitive learning algorithms, or by generating synthetic data points following the distribution of the existing data. While synthetic data generation has recently received a great deal of attention, there are real challenges involved in doing so for high-dime…
University of California San Diego（加利福尼亚大学圣迭戈分校）；CUNY Graduate Center（纽约市立大学研究生中心）；Facebook AI Research（Facebook人工智能研究院）
Recent progress in self-supervised learning has demonstrated promising results in multiple visual tasks. An important ingredient in high-performing self-supervised methods is the use of data augmentation by training models to place different augmented views of the same image nearby in embedding space. However, commonly used augmentation pipelines treat images holistically, ignoring the semantic relevance of parts of an image-e.g. a subject vs. a background-which can lead to the learning of spurious correlations. Our work addresses this problem by investigating a class of simple, yet highly eff…
The University of Tokyo（东京大学）；LPIXEL Inc.（LPIXEL公司）；National Institute of Informatics（国立信息学研究所）；University of Cambridge（剑桥大学）；Keio University（庆应义塾大学）
Despite the success of Convolutional Neural Network-based Computer-Aided Diagnosis research, its clinical applications remain challenging. Accordingly, developing medical Artificial Intelligence (AI) fitting into a clinical environment requires identifying/bridging the gap between AI and Healthcare sides. Since the biggest problem in Medical Imaging lies in data paucity, confirming the clinical relevance for diagnosis of research-proven image augmentation techniques is essential. Therefore, we hold a clinically valuable AI-envisioning workshop among Japanese Medical Imaging experts, physicians…
In this work, we propose a deep learning approach to forecasting state-level COVID-19 trends of weekly cumulative death in the United States (US) and incident cases in Germany. This approach includes a transformer model, an ensemble method, and a data augmentation technique for time series. We arrange the inputs of the transformer in such a way that predictions for different states can attend to the trends of the others. To overcome the issue of scarcity of training data for this COVID-19 pandemic, we have developed a novel data augmentation technique to generate useful data for training. More…
Sliding-Window Reordering with Overlap Averaging: A Simple Time-Domain Augmentation for Multivariate Forecasting
Data Science Group, University of Hildesheim（希尔德斯海姆大学数据科学组）；Information Systems and Machine Learning Lab (ISMLL), University of Hildesheim（希尔德斯海姆大学信息系统与机器学习实验室）
L-GTA: Latent Generative Modeling for Time Series Augmentation
Faculdade de Engenharia da Universidade do Porto（葡萄牙波尔图大学工程学院）；Fraunhofer Portugal AICOS（德国弗劳恩霍夫葡萄牙AICOS研究所）；University of Coimbra（科英布拉大学）；Dalhousie University（达尔豪斯大学）
The University of Tokyo（东京大学）；National Center for Global Health and Medicine（国立国际医疗研究中心）；Kyushu Medical Center（九州医疗中心）
Convolutional Neural Networks (CNNs) can play a key role in Medical Image Analysis under large-scale annotated datasets. However, preparing such massive dataset is demanding. In this context, Generative Adversarial Networks (GANs) can generate realistic but novel samples, and thus effectively cover the real image distribution. In terms of interpolation, the GAN-based medical image augmentation is reliable because medical modalities can display the human body's strong anatomical consistency at fixed position while clearly reflecting inter-subject variability; thus, we propose to use noise-to-im…
Online Data Augmentation for Forecasting with Deep Learning
Faculdade de Engenharia da Universidade do Porto（波尔图大学工程学院）；Laboratory for Artificial Intelligence and Computer Science (LIACC)（人工智能与计算机科学实验室）；INESC TEC；Fraunhofer Portugal AICOS（弗劳恩霍夫葡萄牙AICOS）
Embedding-Space Data Augmentation to Prevent Membership Inference Attacks in Clinical Time Series Forecasting
Department of Computational Linguistics Heidelberg University Germany（计算语言学系海德堡大学）；Department of Computational Linguistics & Interdisciplinary Center for Scientific Computing (IWR) Heidelberg University Germany（计算语言学系与跨学科科学计算中心（IWR）海德堡大学）
University of Georgia（佐治亚大学）；Kennesaw State University（肯尼索州立大学）
Statistical methods such as the Box-Jenkins method for time-series forecasting have been prominent since their development in 1970. Many researchers rely on such models as they can be efficiently estimated and also provide interpretability. However, advances in machine learning research indicate that neural networks can be powerful data modeling techniques, as they can give higher accuracy for a plethora of learning problems and datasets. In the past, they have been tried on time-series forecasting as well, but their overall results have not been significantly better than the statistical model…
Meta-learning and Data Augmentation for Stress Testing Forecasting Models
Faculdade de Engenharia da Universidade do Porto（波尔图大学工程学院）；Laboratory for Artificial Intelligence and Computer Science (LIACC)（人工智能与计算机科学实验室（LIACC））；Fraunhofer Portugal AICOS（弗劳恩霍夫葡萄牙AICOS研究所）
Overcoming Data Limitations in Internet Traffic Forecasting: LSTM Models with Transfer Learning and Wavelet Augmentation
Department of Computer Science, University of Northern British Columbia（北坎伯兰大学计算机科学系）；Department of Computer Science, Western University（西部大学计算机科学系）；Juniper Networks
Relevance-driven Input Dropout: an Explanation-guided Regularization Technique
Fraunhofer Heinrich Hertz Institute, Department of Artificial Intelligence（弗劳恩霍夫海因里希·赫茨研究所人工智能部门）；Lehrstuhl für Angewandte Mathematik, University of Bayreuth（拜罗伊特大学应用数学研究所）；Technische Universität Berlin（柏林技术大学）；BIFOLD – Berlin Institute for the Foundations of Learning and Data（柏林学习与数据基础研究所）；Centre of eXplainable Artificial Intelligence, Technological University Dublin（解释性人工智能中心，都柏林技术大学）
Time series forecasting is an important yet challenging task. Though deep learning methods have recently been developed to give superior forecasting results, it is crucial to improve the interpretability of time series models. Previous interpretation methods, including the methods for general neural networks and attention-based methods, mainly consider the interpretation in the feature dimension while ignoring the crucial temporal dimension. In this paper, we present the series saliency framework for temporal interpretation for multivariate time series forecasting, which considers the forecast…
Dalian University of Technology（大连理工大学）；Federation University Australia（澳大利亚联邦大学）
Graph Augmentation Learning (GAL) provides outstanding solutions for graph learning in handling incomplete data, noise data, etc. Numerous GAL methods have been proposed for graph-based applications such as social network analysis and traffic flow forecasting. However, the underlying reasons for the effectiveness of these GAL methods are still unclear. As a consequence, how to choose optimal graph augmentation strategy for a certain application scenario is still in black box. There is a lack of systematic, comprehensive, and experimentally validated guideline of GAL for scholars. Therefore, in…
Time Series Data Augmentation as an Imbalanced Learning Problem
Faculdade de Engenharia da Universidade do Porto（波尔图大学工程学院）；Laboratory for Artificial Intelligence and Computer Science (LIACC)（人工智能与计算机科学实验室（LIACC））；Notre Dame University（圣母大学）；Fraunhofer Portugal AICOS（弗劳恩霍夫葡萄牙AICOS机构）
Long-term Wind Power Forecasting with Hierarchical Spatial-Temporal Transformer
Sun Yat-sen University（中山大学）；The Hong Kong Polytechnic University（香港理工大学）；Smart Cities Research Institute, The Hong Kong Polytechnic University（香港理工大学智慧城市研究院）；Research Institute, Sun Yat-sen University（中山大学研究院）
