Time series forecasting plays a crucial role in decision-making across various domains, but it presents significant challenges. Recent studies have explored image-driven approaches using computer vision models to address these challenges, often employing lineplots as the visual representation of time series data. In this paper, we propose a novel approach that uses time-frequency spectrograms as the visual representation of time series data. We introduce the use of a vision transformer for multimodal learning, showcasing the advantages of our approach across diverse datasets from different domains. To evaluate its effectiveness, we compare our method against statistical baselines (EMA and ARIMA), a state-of-the-art deep learning-based approach (DeepAR), other visual representations of time series data (lineplot images), and an ablation study on using only the time series as input. Our experiments demonstrate the benefits of utilizing spectrograms as a visual representation for time series data, along with the advantages of employing a vision transformer for simultaneous learning in both the time and frequency domains.
CloudCast: A Satellite-Based Dataset and Baseline for Forecasting Clouds
Aarhus University(奥胡斯大学)
预测云的形成和发展是现代天气预报系统的核心要素。由于云在地球气候系统中的固有作用,不正确的云预测可能导致天气预报整体准确性的重大不确定性。很少有研究从机器学习的角度来解决这个具有挑战性的问题,因为全球缺乏具有许多历史观测的高分辨率数据集。在本文中,我们提出了一种新的基于卫星的数据集称为“Cloudcast”。它由70,080张图像组成,其中包含10种不同的云类型,用于在像素级别上注释大气的多个层。数据集的空间分辨率为928 x 1530像素(每像素3 x 3 km),帧间间隔为15分钟,时间范围为2017-01-01至2018-12-31。所有帧都居中并投影在欧洲上空。为了补充数据集,我们使用当前最先进的视频预测方法进行了评估研究,例如卷积长短期记忆网络,生成对抗网络和基于光流的外推方法。由于视频预测的评估在实践中是困难的,我们的目标是在空间和时间域进行彻底的评估。我们的基准模型显示出令人鼓舞的结果,但有足够的改进空间。据作者所知,这是第一个公开可用的全球规模数据集,具有高时间粒度上的高分辨率云类型。
M4Fog: A Global Multi-Regional, Multi-Modal, and Multi-Stage Dataset for Marine Fog Detection and Forecasting to Bridge Ocean and Atmosphere
Beijing University of Posts and Telecommunications (BUPT)(北京邮电大学);China National Meteorological Centre (NMC)(中国国家气象中心);Tsinghua University(清华大学);Beihang University(北京航空航天大学);Beijing Wuzi University(北京物资学院)
PixelFlowCast: Latent-Free Precipitation Nowcasting via Pixel Mean Flows
Department of Automation, Southeast University(东南大学自动化系);State Key Laboratory of Virtual Reality Technology and Systems, Beihang University(北京航空航天大学虚拟现实技术与系统国家重点实验室)
Representations Before Pixels: Semantics-Guided Hierarchical Video Prediction
Archimedes Athena Research Center(雅典娜研究中心);National Technical University of Athens(雅典国家技术大学);University of Crete(克里特大学);IACM-Forth(希腊研究与技术基金会-应用与计算数学研究所)
Towards Surgical World-Action Modeling: A Preliminary Joint Visual-Trajectory Forecasting for Surgical Motion Planning
University of Macau(澳门大学);University of Macau Advanced Research Institute in Hengqin(澳门大学横琴研究院);The Chinese University of Hong Kong(香港中文大学);The Hong Kong Polytechnic University(香港理工大学);Shanghai Jiao Tong University(上海交通大学);Tongren Hospital(同仁医院);Duke University(杜克大学)
Cross-Resolution Attention Network for High-Resolution PM2.5 Prediction
Institute for Atmospheric and Earth System Research(大气与地球系统研究所);University of Helsinki(赫尔辛基大学);Department of Computational Engineering(计算工程系);LUT University(卢托纳大学);Atmospheric Modelling Centre Lahti (AMC-Lahti)(拉赫蒂大气建模中心(AMC-Lahti));Advanced Micro Devices (AMD)(先进微器件(AMD))
From Remote Sensing to Multiple Time Horizons Forecasts: Transformers Model for CyanoHAB Intensity in Lake Champlain
Vermont Artificial Intelligence Laboratory (VaiL)(佛蒙特人工智能实验室(VaiL));Department of Computer Science, University of Vermont(大学计算机科学系);University of Vermont(佛蒙特大学);Department of Community Development and Applied Economics & Department of Computer Science, University of Vermont(社区发展与应用经济学系及计算机科学系,佛蒙特大学);Department of Geology, University of Vermont(地质学系,佛蒙特大学);Department of Civil and Environmental Engineering, University of Vermont(土木与环境工程系,佛蒙特大学);Vermont EPSCoR (Established Program to Stimulate Competitive Research), University of Vermont(佛蒙特EPSCoR(促进竞争性研究计划),佛蒙特大学);Vermont Department of Environmental Conservation(佛蒙特州环境保护部);Water Quality Solutions, Inc.(水质解决方案公司)
OSMGraphCLIP: Learning Global Location Representations from OpenStreetMap Graphs
Harokopio University of Athens(雅典哈罗科皮奥大学);National Technical University of Athens(雅典国家技术大学);Vienna University of Technology(维也纳技术大学);National Observatory of Athens(雅典国家天文台)
Resolving Spatio-Temporal Entanglement in Video Prediction via Multi-Modal Attention
Indian Institute of Technology (BHU), Varanasi(印度理工学院(巴纳拉斯印度教大学),瓦拉纳西);University of Colorado, Boulder(科罗拉多大学博尔德分校);Erasmus+, Intelligent Field Robotic Systems (IFRoS), University of Girona(伊拉斯谟+,智能现场机器人系统(IFRoS),赫罗纳大学)