Advances in deep learning systems have allowed large models to match or surpass human accuracy on a number of skills such as image classification, basic programming, and standardized test taking. As the performance of the most capable models begin to saturate on tasks where humans already achieve high accuracy, it becomes necessary to benchmark models on increasingly complex abilities. One such task is forecasting the future outcome of events. In this work we describe experiments using a novel dataset of real world events and associated human predictions, an evaluation metric to measure forecasting ability, and the accuracy of a number of different LLM based forecasting designs on the provided dataset. Additionally, we analyze the performance of the LLM forecasters against human predictions and find that models still struggle to make accurate predictions about the future. Our follow-up experiments indicate this is likely due to models' tendency to guess that most events are unlikely to occur (which tends to be true for many prediction datasets, but does not reflect actual forecasting abilities). We reflect on next steps for developing a systematic and reliable approach to studying LLM forecasting.
Embracing Large Language Models in Traffic Flow Forecasting
State Key Laboratory for Multimedia Information Processing, School of Computer Science, PKU-Anker LLM Lab, Peking University（信息处理国家重点实验室、计算机学院、PKU-Anker LLM实验室、北京大学）；Department of Computer Science, University of California, Los Angeles（计算机科学系、加州大学洛杉矶分校）；Carnegie Mellon University（卡内基梅隆大学）；Paul G. Allen School of Computer Science and Engineering, University of Washington（保罗·G·阿伦计算机科学与工程学院、华盛顿大学）
Hidden Leaks in Time Series Forecasting: How Data Leakage Affects LSTM Evaluation Across Configurations and Validation Strategies
King Fahd University of Petroleum \& Minerals, Department of Information；Computer Science, Dhahran, Saudi Arabia Imam Abdulrahman Bin Faisal University, Department of Computer Science, Dammam, Saudi Arabia SDAIA-KFUPM Joint Research Center for Artificial Intelligence, Dhahran, Saudi Arabia
RePST: Language Model Empowered Spatio-Temporal Forecasting via Semantic-Oriented Reprogramming
The Hong Kong University of Science and Technology (Guangzhou)（香港科学与技术大学（广州））；The Hong Kong University of Science and Technology（香港科学与技术大学）；Shandong University（山东大学）；Oxford University（牛津大学）；Beihang University（北航大学）
School of Computer Science, University of Bristol（布里斯托尔大学计算机科学学院）；School of Physics, University of Bristol（布里斯托尔大学物理学院）；School of Engineering Mathematics and Technology, University of Bristol（布里斯托尔大学工程数学与技术学院）
T-LLM: Teaching Large Language Models to Forecast Time Series via Temporal Distillation
State Key Laboratory for Novel Software Technology, Nanjing University（南京大学新型软件技术国家重点实验室）；School of Artificial Intelligence, Nanjing University（南京大学人工智能学院）
Time-LLM: Time Series Forecasting by Reprogramming Large Language Models
Monash University（莫纳什大学）；Ant Group（蚂蚁集团）；IBM Research（IBM研究院）；Griffith University（格里菲斯大学）；Alibaba Group（阿里巴巴集团）；The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））
Advancing Enterprise Spatio-Temporal Forecasting Applications: Data Mining Meets Instruction Tuning of Language Models For Multi-modal Time Series Analysis in Low-Resource Settings
Reprogramming Foundational Large Language Models(LLMs) for Enterprise Adoption for Spatio-Temporal Forecasting Applications: Unveiling a New Era in Copilot-Guided Cross-Modal Time Series Representation Learning
Zero-shot Load Forecasting for Integrated Energy Systems: A Large Language Model-based Framework with Multi-task Learning
School of Automation Science and Engineering, Xi’an Jiaotong University（西安交通大学自动化科学与工程学院）；School of Electronic Information Engineering, Shanghai Dianji University（上海电机学院电子信息工程学院）；School of Energy and Power Engineering, Xi'an Jiaotong University（西安交通大学能源与动力工程学院）；Eindhoven University of Technology（埃因霍温理工大学）；School of Microelectronics, Xi’an Jiaotong University（西安交通大学微电子学院）；State Key Laboratory for Manufacturing System Engineering Lab, Xi’an Jiaotong University（西安交通大学制造系统工程国家重点实验室）
Towards Long-Horizon Vessel Trajectory and Destination Forecasting with Reasoning Large Language Models
Institute of High Performance Computing (IHPC), A*STAR, Singapore（新加坡科技研究局高性能计算研究所）；The Key Laboratory of Road and Traffic Engineering, Ministry of Education, Tongji University（同济大学道路与交通工程教育部重点实验室）；Meituan Inc., Shenzhen, China（美团（深圳））；Centre for Frontier AI Research (CFAR), A*STAR, Singapore（新加坡科技研究局前沿人工智能研究中心）；Nankai University（南开大学）；School of Artificial Intelligence, Jilin University（吉林大学人工智能学院）
Advancing Real-time Pandemic Forecasting Using Large Language Models: A COVID-19 Case Study
Johns Hopkins University（约翰斯·霍普金斯大学）；Mila - Quebec AI Institute（米拉-魁北克人工智能研究所）；Harvard T.H. Chan School of Public Health（哈佛陈曾熙公共卫生学院）；Harvard University（哈佛大学）；Universite de Montréal（蒙特利尔大学）；Duke University（杜克大学）；Johns Hopkins Bloomberg School of Public Health（约翰斯·霍普金斯大学布隆伯格公共卫生学院）
Macroeconomic Forecasting with Large Language Models
School of Economics and Finance, Queen Mary University of London（伦敦大学女王学院经济与金融学院）；School of Business and Economics, Brandeis University（布兰迪大学商学院与经济学学院）
OpenTSLM TeeMoE: A Unified Time-Series Language Model for Forecasting, Contextual Prediction, and Reasoning
Columbia University, USA（哥伦比亚大学）；Stanford University, USA（斯坦福大学）；Aionic Labs, Switzerland（Aionic Labs）；Google Agentic Systems Lab, ETH Zürich, Switzerland（谷歌智能体系统实验室，苏黎世联邦理工学院）；University of Pisa, Italy（比萨大学）；National University of Singapore（新加坡国立大学）
LoFT-LLM: Low-Frequency Time-Series Forecasting with Large Language Models
School of Computer Science and Technology, Harbin Institute of Technology (Shenzhen)（计算机科学与技术学院，哈尔滨工业大学（深圳））；The Chinese University of Hong Kong（香港中文大学）
