In the early observation period of a time series, there might be only a few historic observations available to learn a model. However, in cases where an existing prior set of datasets is available, Meta learning methods can be applicable. In this paper, we devise a Meta learning method that exploits samples from additional datasets and learns to augment time series through adversarial learning as an auxiliary task for the target dataset. Our model (FEML), is equipped with a shared Convolutional backbone that learns features for varying length inputs from different datasets and has dataset specific heads to forecast for different output lengths. We show that FEML can meta learn across datasets and by additionally learning on adversarial generated samples as auxiliary samples for the target dataset, it can improve the forecasting performance compared to single task learning, and various solutions adapted from Joint learning, Multi-task learning and classic forecasting baselines.
UC San Diego（加州大学圣地亚哥分校）；Northeastern University（东北大学）
Current deep learning models for dynamics forecasting struggle with generalization. They can only forecast in a specific domain and fail when applied to systems with different parameters, external forces, or boundary conditions. We propose a model-based meta-learning method called DyAd which can generalize across heterogeneous domains by partitioning them into different tasks. DyAd has two parts: an encoder which infers the time-invariant hidden features of the task with weak supervision, and a forecaster which learns the shared dynamics of the entire domain. The encoder adapts and controls th…
SSMT: Few-Shot Traffic Forecasting with Single Source Meta-Transfer
Sungkyunkwan University（成均馆大学）；University of California, Riverside（加州大学河滨分校）；Center for Computational & Data Sciences, Independent University, Bangladesh（孟加拉国独立大学计算与数据科学中心）
Lancaster University（兰卡斯特大学）；Technical University of Ostrava（俄斯特拉发技术大学）；Sharif University of Technology（谢里夫理工大学）
One of the challenging questions in time series forecasting is how to find the best algorithm. In recent years, a recommender system scheme has been developed for time series analysis using a meta-learning approach. This system selects the best forecasting method with consideration of the time series characteristics. In this paper, we propose a novel approach to focusing on some of the unanswered questions resulting from the use of meta-learning in time series forecasting. Therefore, three main gaps in previous works are addressed including, analyzing various subsets of top forecasters as inpu…
This paper presents a meta-learning based, automatic distribution system load forecasting model selection framework. The framework includes the following processes: feature extraction, candidate model labeling, offline training, and online model recommendation. Using user load forecasting needs as input features, multiple meta-learners are used to rank the available load forecast models based on their forecasting accuracy. Then, a scoring-voting mechanism weights recommendations from each meta-leaner to make the final recommendations. Heterogeneous load forecasting tasks with different tempora…
APPCAIR；BITS Pilani, K K Birla Goa Campus（皮拉尼比尔拉理工学院果阿校区）；TCS Research（塔塔咨询服务公司研究院）
Deep-learning techniques have been successfully used for time-series forecasting and have often shown superior performance on many standard benchmark datasets as compared to traditional techniques. Here we present a comprehensive and comparative study of performance of deep-learning techniques for forecasting prices in financial markets. We benchmark state-of-the-art deep-learning baselines, such as NBeats, etc., on data from currency as well as stock markets. We also generate synthetic data using a fuzzy-logic based model of demand driven by technical rules such as moving averages, which are …
Tailored Forecasting from Short Time Series via Meta-learning
University of Maryland（马里兰大学）；Institute for Research in Electronics and Applied Physics（电子与应用物理研究所）；Department of Electrical and Computer Engineering（电气与计算机工程系）；Potomac Research LLC（波托马克研究公司）；Institute for Physical Science and Technology（物理科学与技术研究所）；Santa Fe Institute（圣菲研究所）
Techniques of hybridisation and ensemble learning are popular model fusion techniques for improving the predictive power of forecasting methods. With limited research that instigates combining these two promising approaches, this paper focuses on the utility of the Exponential-Smoothing-Recurrent Neural Network (ES-RNN) in the pool of base models for different ensembles. We compare against some state of the art ensembling techniques and arithmetic model averaging as a benchmark. We experiment with the M4 forecasting data set of 100,000 time-series, and the results show that the Feature-based F…
Meta-learning and Data Augmentation for Stress Testing Forecasting Models
Faculdade de Engenharia da Universidade do Porto（波尔图大学工程学院）；Laboratory for Artificial Intelligence and Computer Science (LIACC)（人工智能与计算机科学实验室（LIACC））；Fraunhofer Portugal AICOS（弗劳恩霍夫葡萄牙AICOS研究所）
Delft University of Technology（代尔夫特理工大学）；University of Copenhagen（哥本哈根大学）
Free-standing social conversations constitute a yet underexplored setting for human behavior forecasting. While the task of predicting pedestrian trajectories has received much recent attention, an intrinsic difference between these settings is how groups form and disband. Evidence from social psychology suggests that group members in a conversation explicitly self-organize to sustain the interaction by adapting to one another's behaviors. Crucially, the same individual is unlikely to adapt similarly across different groups; contextual factors such as perceived relationships, attraction, rappo…
Stanford University（斯坦福大学）；AWS AI Labs（AWS人工智能实验室）
Meta-forecasting is a newly emerging field which combines meta-learning and time series forecasting. The goal of meta-forecasting is to train over a collection of source time series and generalize to new time series one-at-a-time. Previous approaches in meta-forecasting achieve competitive performance, but with the restriction of training a separate model for each sampling frequency. In this work, we investigate meta-forecasting over different sampling frequencies, and introduce a new model, the Continuous Frequency Adapter (CFA), specifically designed to learn frequency-invariant representati…
The contribution of this work is twofold: (1) We introduce a collection of ensemble methods for time series forecasting to combine predictions from base models. We demonstrate insights on the power of ensemble learning for forecasting, showing experiment results on about 16000 openly available datasets, from M4, M5, M3 competitions, as well as FRED (Federal Reserve Economic Data) datasets. Whereas experiments show that ensembles provide a benefit on forecasting results, there is no clear winning ensemble strategy (plus hyperparameter configuration). Thus, in addition, (2), we propose a meta-le…
STEMO: Early Spatio-temporal Forecasting with Multi-Objective Reinforcement Learning
Data61, CSIRO（Data61联邦科学与工业研究组织）；RMIT University（皇家墨尔本理工大学）；Xidian University（西安电子科技大学）；Zhejiang University（浙江大学）；University of New South Wales（新南威尔士大学）
Heterogeneity-Informed Meta-Parameter Learning for Spatiotemporal Time Series Forecasting
Southern University of Science and Technology（南方科技大学）；The University of Tokyo（东京大学）；Hong Kong University of Science and Technology（香港科技大学）；Squirrel AI（松鼠AI）；Jilin University（吉林大学）
Institute for Intelligent Systems（智能系统研究所）；University of Johannesburg（约翰内斯堡大学）
Meta-learning, decision fusion, hybrid models, and representation learning are topics of investigation with significant traction in time-series forecasting research. Of these two specific areas have shown state-of-the-art results in forecasting: hybrid meta-learning models such as Exponential Smoothing - Recurrent Neural Network (ES-RNN) and Neural Basis Expansion Analysis (N-BEATS) and feature-based stacking ensembles such as Feature-based FORecast Model Averaging (FFORMA). However, a unified taxonomy for model fusion and an empirical comparison of these hybrid and feature-based stacking ense…
Dalian University of Technology（大连理工大学）；Microsoft Research（微软研究院）；LinkedIn Corporation（领英公司）
User engagement prediction plays a critical role for designing interaction strategies to grow user engagement and increase revenue in online social platforms. Through the in-depth analysis of the real-world data from the world's largest professional social platforms, i.e., LinkedIn, we find that users expose diverse engagement patterns, and a major reason for the differences in user engagement patterns is that users have different intents. That is, people have different intents when using LinkedIn, e.g., applying for jobs, building connections, or checking notifications, which shows quite diff…
Gaussian Process Latent Variable Modeling for Few-shot Time Series Forecasting
Department of Computer Science, Aalborg University（奥尔堡大学计算机科学系）；Department of Computer Science, East China Normal University（华东师范大学计算机科学系）；Huawei Cloud Database Innovation Lab（华为云数据库创新实验室）；University of Electronic Science and Technology of China（电子科技大学）
IIT（意大利理工学院）；UCL（伦敦大学学院）；Amazon Web Services（亚马逊云服务）
While classical time series forecasting considers individual time series in isolation, recent advances based on deep learning showed that jointly learning from a large pool of related time series can boost the forecasting accuracy. However, the accuracy of these methods suffers greatly when modeling out-of-sample time series, significantly limiting their applicability compared to classical forecasting methods. To bridge this gap, we adopt a meta-learning view of the time series forecasting problem. We introduce a novel forecasting method, called Meta Global-Local Auto-Regression (Meta-GLAR), t…
Phillips Academy Andover（菲利普斯学院安多弗）；Pennsylvania State University（宾夕法尼亚州立大学）
When faced with self-regulation challenges, children have been known the use their language to inhibit their emotions and behaviors. Yet, to date, there has been a critical lack of evidence regarding what patterns in their speech children use during these moments of frustration. In this paper, eXtreme Gradient Boosting, Random Forest, Long Short-Term Memory Recurrent Neural Networks, and Elastic Net Regression, have all been used to forecast these language patterns in children. Based on the results of a comparative analysis between these methods, the study reveals that when dealing with high-d…
Can meta-learning discover generic ways of processing time series (TS) from a diverse dataset so as to greatly improve generalization on new TS coming from different datasets? This work provides positive evidence to this using a broad meta-learning framework which we show subsumes many existing meta-learning algorithms. Our theoretical analysis suggests that residual connections act as a meta-learning adaptation mechanism, generating a subset of task-specific parameters based on a given TS input, thus gradually expanding the expressive power of the architecture on-the-fly. The same mechanism i…
Kaunas University of Technology（考纳斯理工大学）；Rivile（Rivile私人有限责任公司）
Amounts of historical data collected increase and business intelligence applicability with automatic forecasting of time series are in high demand. While no single time series modeling method is universal to all types of dynamics, forecasting using an ensemble of several methods is often seen as a compromise. Instead of fixing ensemble diversity and size, we propose to predict these aspects adaptively using meta-learning. Meta-learning here considers two separate random forest regression models, built on 390 time-series features, to rank 22 univariate forecasting methods and recommend ensemble…
We consider a general class of forecasting protocols, called "linear protocols", and discuss several important special cases, including multi-class forecasting. Forecasting is formalized as a game between three players: Reality, whose role is to generate observations; Forecaster, whose goal is to predict the observations; and Skeptic, who tries to make money on any lack of agreement between Forecaster's predictions and the actual observations. Our main mathematical result is that for any continuous strategy for Skeptic in a linear protocol there exists a strategy for Forecaster that does not a…
The University of Tokyo（东京大学）；Toyota Motor Corporation（丰田汽车公司）
Traffic forecasting as a canonical task of multivariate time series forecasting has been a significant research topic in AI community. To address the spatio-temporal heterogeneity and non-stationarity implied in the traffic stream, in this study, we propose Spatio-Temporal Meta-Graph Learning as a novel Graph Structure Learning mechanism on spatio-temporal data. Specifically, we implement this idea into Meta-Graph Convolutional Recurrent Network (MegaCRN) by plugging the Meta-Graph Learner powered by a Meta-Node Bank into GCRN encoder-decoder. We conduct a comprehensive evaluation on two bench…
Forecasting Anomaly Precursors via Uncertainty-Aware Time-Series Ensembles
Department of Industrial & Management Engineering, Korea University（韩国大学工业与管理工程系）；Department of Industrial Engineering, Seoul National University（首尔国立大学工业工程系）；LG CNS
Massachusetts Institute of Technology（麻省理工学院）；Argonne National Laboratory（阿贡国家实验室）
Accurately forecasting the weather is a key requirement for climate change mitigation. Data-driven methods offer the ability to make more accurate forecasts, but lack interpretability and can be expensive to train and deploy if models are not carefully developed. Here, we make use of two historical climate data sets and tools from machine learning, to accurately predict temperature fields. Furthermore, we are able to use low fidelity models that are cheap to train and evaluate, to selectively avoid expensive high fidelity function evaluations, as well as uncover seasonal variations in predicti…
Towards Accurate and Interpretable Time-series Forecasting: A Polynomial Learning Approach
Center for Intelligent Decision-Making（智能决策中心）；Machine Learning, School of Management, Xi'an Jiaotong University, Xi'an, China（机器学习，管理学院，西安交通大学，中国）；The 39th Research Institute of China Electronics Technology Group Corporation, Xi'an, China（中国电子科技集团第三十九研究所，中国）；Shenzhen Research Institute of Big Data, ShenZhen, China（深圳大数据研究院，中国）
Are We Really Benchmarking Forecasting Models? The Impact of Preprocessing on Time Series Performance
LIVIA, Department of Software and IT Engineering（LIVIA，软件与IT工程系）；École de technologie supérieure (ÉTS)（高等技术学院）；Center for Informatics (CIn)（信息学中心）；Universidade Federal de Pernambuco (UFPE)（伯南布哥联邦大学）
When creating text classification systems, one of the major bottlenecks is the annotation of training data. Active learning has been proposed to address this bottleneck using stopping methods to minimize the cost of data annotation. An important capability for improving the utility of stopping methods is to effectively forecast the performance of the text classification models. Forecasting can be done through the use of logarithmic models regressed on some portion of the data as learning is progressing. A critical unexplored question is what portion of the data is needed for accurate forecasti…
Real-time probabilistic tsunami forecasting via generative AI
Fujitsu Research, Fujitsu Limited（富士通研究所，富士通株式会社）；Earthquake Research Institute, The University of Tokyo（东京大学地震研究所）；International Research Institute of Disaster Science, Tohoku University（东北大学国际灾害科学研究所）
