arXivDaily arXiv每日学术速递 周一至周五更新
北美退伍军人节(Veterans Day),12 日无更新
arXiv 2511.02458cs.CLcs.CEecon.GNq-fin.EC
收藏

为政策而提示:利用合成 LLM 人物角色预测宏观经济情景

Prompting for Policy: Forecasting Macroeconomic Scenarios with Synthetic LLM Personas

Banca d’Italia(意大利银行)

Giulia Iadisernia, Carolina Camassa

AI总结:

本文利用 PersonaHub 中 2,368 个经济学人物角色提示 GPT-4o 复现欧洲央行专业预测者调查,发现其预测准确度与人类专家相当且样本外仍具竞争力,但人物角色提示本身并无显著增益,可省略以降低成本。

摘要:

我们评估基于人物角色的提示是否能提升大语言模型(LLM)在宏观经济预测任务上的表现。我们使用 PersonaHub 语料库中的 2,368 个经济学相关人物角色,提示 GPT-4o 在 50 个季度轮次(2013-2025 年)中复现欧洲央行专业预测者调查(ECB Survey of Professional Forecasters)。我们将人物角色提示的预测与人类专家小组进行比较,涵盖四个目标变量(HICP、核心 HICP、GDP 增长、失业率)和四个预测期限。我们还将结果与 100 个不含人物角色描述的基线预测进行比较,以隔离其影响。我们报告两个主要发现。首先,GPT-4o 与人类预测者达到了非常相似的准确度水平,差异在统计上显著但在实际意义上不大。我们在 2024-2025 年数据上的样本外评估表明,GPT-4o 能够在未见事件上保持有竞争力的预测表现,尽管与样本内时期相比存在显著差异。其次,我们的消融实验显示人物角色描述没有带来可衡量的预测优势,表明这些提示组件可以被省略,以在不牺牲准确度的情况下降低计算成本。我们的结果提供证据表明,如果提供相关上下文数据,GPT-4o 即使在样本外宏观经济事件上也能达到有竞争力的预测准确度,同时揭示出与人类小组相比,多样化的提示产生了非常同质化的预测。

英文摘要:

We evaluate whether persona-based prompting improves Large Language Model (LLM) performance on macroeconomic forecasting tasks. Using 2,368 economics-related personas from the PersonaHub corpus, we prompt GPT-4o to replicate the ECB Survey of Professional Forecasters across 50 quarterly rounds (2013-2025). We compare the persona-prompted forecasts against the human experts panel, across four target variables (HICP, core HICP, GDP growth, unemployment) and four forecast horizons. We also compare the results against 100 baseline forecasts without persona descriptions to isolate its effect. We report two main findings. Firstly, GPT-4o and human forecasters achieve remarkably similar accuracy levels, with differences that are statistically significant yet practically modest. Our out-of-sample evaluation on 2024-2025 data demonstrates that GPT-4o can maintain competitive forecasting performance on unseen events, though with notable differences compared to the in-sample period. Secondly, our ablation experiment reveals no measurable forecasting advantage from persona descriptions, suggesting these prompt components can be omitted to reduce computational costs without sacrificing accuracy. Our results provide evidence that GPT-4o can achieve competitive forecasting accuracy even on out-of-sample macroeconomic events, if provided with relevant context data, while revealing that diverse prompts produce remarkably homogeneous forecasts compared to human panels.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑