AI-Augmented Predictions: LLM Assistants Improve Human Forecasting Accuracy
London School of Economics and Political Science（伦敦政治经济学院）；Massachusetts Institute of Technology（麻省理工学院）；Federal Reserve Bank of Chicago（芝加哥联邦储备银行）；University of California San Diego（加利福尼亚大学圣迭戈分校）；University of Pennsylvania（宾夕法尼亚大学）
Philipp Schoenegger, Peter S. Park, Ezra Karger, Sean Trott, Philip E. Tetlock
备注：
22 pages pages (main text comprised of 19 pages, appendix comprised of three pages). 10 visualizations in the main text (four figures, six tables), three additional figures in the appendix
Large language models (LLMs) match and sometimes exceeding human performance in many domains. This study explores the potential of LLMs to augment human judgement in a forecasting task. We evaluate the effect on human forecasters of two LLM assistants: one designed to provide high-quality ("superforecasting") advice, and the other designed to be overconfident and base-rate neglecting, thus providing noisy forecasting advice. We compare participants using these assistants to a control group that received a less advanced model that did not provide numerical predictions or engaged in explicit discussion of predictions. Participants (N = 991) answered a set of six forecasting questions and had the option to consult their assigned LLM assistant throughout. Our preregistered analyses show that interacting with each of our frontier LLM assistants significantly enhances prediction accuracy by between 24 percent and 28 percent compared to the control group. Exploratory analyses showed a pronounced outlier effect in one forecasting item, without which we find that the superforecasting assistant increased accuracy by 41 percent, compared with 29 percent for the noisy assistant. We further examine whether LLM forecasting augmentation disproportionately benefits less skilled forecasters, degrades the wisdom-of-the-crowd by reducing prediction diversity, or varies in effectiveness with question difficulty. Our data do not consistently support these hypotheses. Our results suggest that access to a frontier LLM assistant, even a noisy one, can be a helpful decision aid in cognitively demanding tasks compared to a less powerful model that does not provide specific forecasting advice. However, the effects of outliers suggest that further research into the robustness of this pattern is needed.
DermETAS-SNA LLM: A Dermatology Focused Evolutionary Transformer Architecture Search with StackNet Augmented LLM Assistant
Data Science Program, Worcester Polytechnic Institute, Massachusetts, USA（沃斯特理工学院数据科学项目，马萨诸塞州，美国）；Computer Science Department, Worcester Polytechnic Institute, Massachusetts, USA（沃斯特理工学院计算机科学系，马萨诸塞州，美国）；PASE Advisory Group, New Jersey, USA（PASE顾问小组，新泽西州，美国）
TyphoFormer: Language-Augmented Transformer for Accurate Typhoon Track Forecasting
Florida State University（佛罗里达州立大学）；FAMU-FSU College of Engineering（FAMU-FSU 工程学院）；University of Southern California（南加州大学）；University of Maryland（马里兰大学）
AST-GIN: Attribute-Augmented Spatial-Temporal Graph Informer Network for Electric Vehicle Charging Station Availability Forecasting
Nanyang Technological University（南洋理工大学）；Institute for Infocomm Research (I2R), Agency for Science, Technology and Research (ASTAR)（新加坡科技研究局资讯通信研究院（I2R））
GlyRAG: Context-Aware Retrieval-Augmented Framework for Blood Glucose Forecasting
College of Health Solutions, Arizona State University（亚利桑那州立大学健康解决方案学院）；School of Computing and Augmented Intelligence, Arizona State University（亚利桑那州立大学计算与增强智能学院）
Domain-Specific Improvement on Psychotherapy Chatbot Using Assistant
Czech Technical University in Prague（布拉格捷克技术大学）；National Institute of Mental Health in Prague（布拉格国家精神卫生研究所）；The University of Hong Kong（香港大学）；Shenzhen Mental Health Centre（深圳市精神卫生中心）
Retrieval-Augmented Diffusion Models for Time Series Forecasting
School of Intelligence Science and Technology, Peking University（北京大学智能科学与技术学院）；Institute of Medical Technology, Health Science Center of Peking University（北京大学健康科学中心医学技术研究所）；National Key Laboratory of General Artificial Intelligence, Peking University（北京大学通用人工智能全国重点实验室）
TS-RAG: Retrieval-Augmented Generation based Time Series Foundation Models are Stronger Zero-Shot Forecaster
School of Computing, University of Connecticut（康奈尔大学计算机学院）；Department of Machine Learning Research, Morgan Stanley（摩根士丹利机器学习研究部）；Ant Group（蚂蚁集团）；TWG Global（TWG全球）
