A forecasting ensemble consisting of a diverse range of estimators for both local and global univariate forecasting, in particular MQ-CNN,DeepAR, Prophet, NPTS, ARIMA and ETS, can be used to make forecasts for a variety of problems. This paper delves into the aspect of adding different hyperparameter optimization strategies to the deep learning models in such a setup (DeepAR and MQ-CNN), exploring the trade-off between added training cost and the increase in accuracy for different configurations. It shows that in such a setup, adding hyperparameter optimization can lead to performance improvements, with the final setup having a 9.9 % percent accuracy improvement with respect to the avg-wQL over the baseline ensemble without HPO, accompanied by a 65.8 % increase in end-to-end ensemble latency. This improvement is based on an empirical analysis of combining the ensemble pipeline with different tuning strategies, namely Bayesian Optimisation and Hyperband and different configurations of those strategies. In the final configuration, the proposed combination of ensemble learning and HPO outperforms the state of the art commercial AutoML forecasting solution, Amazon Forecast, with a 3.5 % lower error and 16.0 % lower end-to-end ensemble latency.
University of Georgia（佐治亚大学）；Kennesaw State University（肯尼索州立大学）
Statistical methods such as the Box-Jenkins method for time-series forecasting have been prominent since their development in 1970. Many researchers rely on such models as they can be efficiently estimated and also provide interpretability. However, advances in machine learning research indicate that neural networks can be powerful data modeling techniques, as they can give higher accuracy for a plethora of learning problems and datasets. In the past, they have been tried on time-series forecasting as well, but their overall results have not been significantly better than the statistical model…
University of Toronto（多伦多大学）；University of Wisconsin（威斯康星大学）；IIT Bombay（印度理工学院孟买分校）；Carnegie Mellon University（卡内基梅隆大学）；Stanford University（斯坦福大学）
The challenge that climate change poses to humanity has spurred a rapidly developing field of artificial intelligence research focused on climate change applications. The climate change AI (CCAI) community works on a diverse, challenging set of problems which often involve physics-constrained ML or heterogeneous spatiotemporal data. It would be desirable to use automated machine learning (AutoML) techniques to automatically find high-performing architectures and hyperparameters for a given dataset. In this work, we benchmark popular AutoML libraries on three high-leverage CCAI applications: cl…
Enhancing Prediction and Analysis of UK Road Traffic Accident Severity Using AI: Integration of Machine Learning, Econometric Techniques, and Time Series Forecasting in Public Health Research
Hyperparameter optimization (HPO) plays a central role in the automated machine learning (AutoML). It is a challenging task as the response surfaces of hyperparameters are generally unknown, hence essentially a global optimization problem. This paper reformulates HPO as a computer experiment and proposes a novel sequential uniform design (SeqUD) strategy with three-fold advantages: a) the hyperparameter space is adaptively explored with evenly spread design points, without the need of expensive meta-modeling and acquisition optimization; b) the batch-by-batch design points are sequentially gen…
Leibniz University Hannover（汉诺威莱布尼茨大学）；Ludwig-Maximilian University（路德维希-马克西米利安大学）；Fraunhofer Institute for Integrated Circuits (IIS)（弗劳恩霍夫集成电路研究所）；University of Freiburg（弗莱堡大学）；Bosch Center for Artificial Intelligence（博世人工智能中心）
Recent years have witnessed tremendously improved efficiency of Automated Machine Learning (AutoML), especially Automated Deep Learning (AutoDL) systems, but recent work focuses on tabular, image, or NLP tasks. So far, little attention has been paid to general AutoDL frameworks for time series forecasting, despite the enormous success in applying different novel architectures to such tasks. In this paper, we propose an efficient approach for the joint optimization of neural architecture and hyperparameters of the entire data processing pipeline for time series forecasting. In contrast to commo…
Karlsruhe Institute of Technology（卡尔斯鲁厄理工学院）；RWTH Aachen University（亚琛工业大学）；Forschungszentrum Jülich GmbH（于利希研究中心有限公司）
Time series forecasting is fundamental for various use cases in different domains such as energy systems and economics. Creating a forecasting model for a specific use case requires an iterative and complex design process. The typical design process includes the five sections (1) data pre-processing, (2) feature engineering, (3) hyperparameter optimization, (4) forecasting method selection, and (5) forecast ensembling, which are commonly organized in a pipeline structure. One promising approach to handle the ever-growing demand for time series forecasts is automating this design process. The p…
CoFEH: LLM-driven Feature Engineering Empowered by Collaborative Bayesian Hyperparameter Optimization
School of CS \& Key Lab of High Confidence Software Technologies (MOE), Peking University Beijing China；School of CS \& Beijing Key Laboratory of Software；Hardware Cooperative Artificial Intelligence Systems, Peking University Beijing China；School of CS \& Key Lab of High Confidence Software Technologies (MOE), Peking University；Hardware Cooperative Artificial Intelligence Systems, Peking University
Simula Research Laboratory（Simula研究实验室）；University of Thessaly（色萨利大学）；University of Oslo（奥斯陆大学）；Simula Metropolitan（Simula大都会中心）
Bandwidth forecasting in Mobile Broadband (MBB) networks is a challenging task, particularly when coupled with a degree of mobility. In this work, we introduce HINDSIGHT++, an open-source R-based framework for bandwidth forecasting experimentation in MBB networks with Long Short Term Memory (LSTM) networks. We instrument HINDSIGHT++ following an Automated Machine Learning (AutoML) paradigm to first, alleviate the burden of data preprocessing, and second, enhance performance related aspects. We primarily focus on bandwidth forecasting for Fifth Generation (5G) networks. In particular, we levera…
TSGym: Design Choices for Deep Multivariate Time-Series Forecasting
AI Lab, Shanghai University of Finance and Economics（上海财经大学金融学院人工智能实验室）；Ant Group（蚂蚁集团）；MoE Key Laboratory of Interdisciplinary Research of Computation and Economics（计算与经济学交叉学科关键实验室）
Provably Reduced Sample Cost in Prior-Guided Hyperparameter Optimization
Leibniz University Hanover（莱比锡大学汉诺威分校）；University of Bielefeld（比勒菲尔德大学）；Institute of Artificial Intelligence, Leibniz University Hanover（人工智能研究所，莱比锡大学汉诺威分校）；L3S Research Center Hanover（汉诺威L3S研究中心）
Aimsun SLU（Aimsun SLU公司）；DeustoTech（德乌斯托技术中心）；Faculty of Engineering, University of Deusto（德乌斯托大学工程学院）；University of Deusto（德乌斯托大学）；IKERBASQUE, Basque Foundation for Science（巴斯克科学基金会IKERBASQUE）；University of Granada（格拉纳达大学）
Intelligent Transportation Systems are producing tons of hardly manageable traffic data, which motivates the use of Machine Learning (ML) for data-driven applications, such as Traffic Forecasting (TF). TF is gaining relevance due to its ability to mitigate traffic congestion by forecasting future traffic states. However, TF poses one big challenge to the ML paradigm, known as the Model Selection Problem (MSP): deciding the most suitable combination of data preprocessing techniques and ML method for traffic data collected under different transportation circumstances. In this context, Automated …
An AutoML Framework using AutoGluonTS for Forecasting Seasonal Extreme Temperatures
Facultad de Ingeniería, Universidad de la República（拉普拉塔大学工程学院）；Departamento de Física de la Tierra y Astrofísica, Universidad Complutense de Madrid（马德里complutense大学地球与天体物理系）；Faculty of Exact and Natural Sciences, Universidad de Buenos Aires (UBA)（布宜诺斯艾利斯大学（UBA）精确与自然科学学院）；Technical University of Denmark（丹麦技术大学）；INRIA（法国国家信息与自动化技术研究所）
