ADATIME: A Benchmarking Suite for Domain Adaptation on Time Series Data
Institute for Infocomm Research, A*STAR(新加坡科技研究局资讯通信研究院);Centre for Frontier AI Research, A*STAR(新加坡科技研究局前沿人工智能研究中心);Nanyang Technological University(南洋理工大学)
Mohamed Ragab, Emadeldeen Eldele, Wee Ling Tan, Chuan-Sheng Foo, Zhenghua Chen, Min Wu, Chee-Keong Kwoh, Xiaoli Li
备注:
Accepted in the ACM Transactions on Knowledge Discovery from Data (TKDD)
Unsupervised domain adaptation methods aim to generalize well on unlabeled test data that may have a different (shifted) distribution from the training data. Such methods are typically developed on image data, and their application to time series data is less explored. Existing works on time series domain adaptation suffer from inconsistencies in evaluation schemes, datasets, and backbone neural network architectures. Moreover, labeled target data are often used for model selection, which violates the fundamental assumption of unsupervised domain adaptation. To address these issues, we develop a benchmarking evaluation suite (AdaTime) to systematically and fairly evaluate different domain adaptation methods on time series data. Specifically, we standardize the backbone neural network architectures and benchmarking datasets, while also exploring more realistic model selection approaches that can work with no labeled data or just a few labeled samples. Our evaluation includes adapting state-of-the-art visual domain adaptation methods to time series data as well as the recent methods specifically developed for time series data. We conduct extensive experiments to evaluate 11 state-of-the-art methods on five representative datasets spanning 50 cross-domain scenarios. Our results suggest that with careful selection of hyper-parameters, visual domain adaptation methods are competitive with methods proposed for time series domain adaptation. In addition, we find that hyper-parameters could be selected based on realistic model selection approaches. Our work unveils practical insights for applying domain adaptation methods on time series data and builds a solid foundation for future works in the field. The code is available at \href{https://github.com/emadeldeen24/AdaTime}{github.com/emadeldeen24/AdaTime}.
LMU Munich(慕尼黑大学);University of Freiburg(弗莱堡大学);Eindhoven University of Technology(埃因霍温理工大学);Bosch Center for Artificial Intelligence(博世人工智能中心);TU Dortmund University(多特蒙德工业大学);Federal Technology University Paraná (UTFPR)(巴拉那联邦科技大学);Leiden University(莱顿大学)
COCO: The Large Scale Black-Box Optimization Benchmarking (bbob-largescale) Test Suite
Inria research centre Saclay(法国国家信息与自动化研究所萨克雷研究中心);CMAP, Ecole Polytechnique(巴黎综合理工学院应用数学中心);Hanoi National University of Education(河内国家教育大学);Jožef Stefan Institute(约瑟夫·斯特凡研究所)
Optimizing with Low Budgets: a Comparison on the Black-box Optimization Benchmarking Suite and OpenAI Gym
Leiden University(莱顿大学);Meta AI Research(Meta AI 研究院);Technical University of Munich(慕尼黑工业大学);LIP6, CNRS, Sorbonne Université(索邦大学国家科学研究中心LIP6实验室);Universitat Pompeu Fabra(庞培法布拉大学);Université d’Antananarivo(塔那那利佛大学)
Black-Box Optimization Revisited: Improving Algorithm Selection Wizards through Massive Benchmarking
Facebook AI Research(Facebook AI研究院);The Chinese University of Hong Kong(香港中文大学);Université Paris-Saclay(巴黎萨克雷大学);Université Clermont Auvergne(克莱蒙奥弗涅大学);Sorbonne Université(索邦大学)