Directly Forecasting Belief for Reinforcement Learning with Delays
University of Southampton(南安普顿大学);Northwestern University(西北大学);National Taiwan University(台湾大学);The Swiss AI Lab IDSIA/USI/SUPSI(瑞士人工智能实验室 IDSIA/USI/SUPSI)
Qingyuan Wu, Yuhui Wang, Simon Sinong Zhan, Yixuan Wang, Chung-Wei Lin, Chen Lv, Qi Zhu, Jürgen Schmidhuber, Chao Huang
Reinforcement learning (RL) with delays is challenging as sensory perceptions lag behind the actual events: the RL agent needs to estimate the real state of its environment based on past observations. State-of-the-art (SOTA) methods typically employ recursive, step-by-step forecasting of states. This can cause the accumulation of compounding errors. To tackle this problem, our novel belief estimation method, named Directly Forecasting Belief Transformer (DFBT), directly forecasts states from observations without incrementally estimating intermediate states step-by-step. We theoretically demonstrate that DFBT greatly reduces compounding errors of existing recursively forecasting methods, yielding stronger performance guarantees. In experiments with D4RL offline datasets, DFBT reduces compounding errors with remarkable prediction accuracy. DFBT's capability to forecast state sequences also facilitates multi-step bootstrapping, thus greatly improving learning efficiency. On the MuJoCo benchmark, our DFBT-based method substantially outperforms SOTA baselines. Code is available at https://github.com/QingyuanWuNothing/DFBT.
STEMO: Early Spatio-temporal Forecasting with Multi-Objective Reinforcement Learning
Data61, CSIRO(Data61联邦科学与工业研究组织);RMIT University(皇家墨尔本理工大学);Xidian University(西安电子科技大学);Zhejiang University(浙江大学);University of New South Wales(新南威尔士大学)
Unveiling Delay Effects in Traffic Forecasting: A Perspective from Spatial-Temporal Delay Differential Equations
Computer Network Information Center, Chinese Academy of Sciences(中国科学院计算机网络信息中心);Peking University(北京大学);Institute of Zoology, Chinese Academy of Sciences(中国科学院动物研究所);Terminus Group(特斯联科技集团);University of Chinese Academy of Sciences(中国科学院大学)
ReLaMix: Residual Latency-Aware Mixing for Delay-Robust Financial Time-Series Forecasting
Lanzhou University(兰州大学);Brandeis University(品牌斯大学);Shenzhen University(深圳大学);Central University of Finance and Economics(中央财经大学);Central China Normal University(中央师范大学)
Massachusetts Institute of Technology(麻省理工学院);Woods Hole Oceanographic Institution(伍兹霍尔海洋研究所);Boston University(波士顿大学);Atmospheric and Environmental Research(大气与环境研究公司);University of Toronto(多伦多大学);Microsoft Research New England(微软新英格兰研究院);Instituto de Matemática Pura e Aplicada(纯数学与应用数学研究所)
Forecasting MBTA Transit Dynamics: A Performance Benchmarking of Statistical and Machine Learning Models
Sunset High School(Sunset 高中);Woodbridge High School(Woodbridge 高中);Western Academy of Beijing(北京西方学院);Computer Science Department, Metropolitan College(Metropolitan College 计算机科学系)
Institute of Industrial Science, The University of Tokyo(东京大学工业科学研究所);Kozo Keikaku Engineering Inc.(构计工程株式会社);International Research Center for Neurointelligence (WPI-IRCN), The University of Tokyo(东京大学国际神经智能研究中心(WPI-IRCN));Mathematics and Informatics Center, The University of Tokyo(东京大学数学与信息学中心)