虚假日期测试：我们能否信任LLM在宏观经济预测中的样本内准确性？
Fake Date Tests: Can We Trust In-sample Accuracy of LLMs in Macroeconomic Forecasting?
AI总结：
本文通过设计虚假日期测试，检验LLM在宏观经济预测中的样本内准确性是否能推广到样本外表现，发现现代LLM存在样本内预测的先行偏见。
摘要：
大型语言模型（LLM）是一种经济学家在实证研究中开始应用的机器学习工具。一种应用是使用LLM进行宏观经济预测的回测，尽管它们是在用于估计其预测性能的相同数据上训练的。这些样本内准确性结果能否推广到模型的样本外表现？为回答这个问题，我们开发了一类提示敏感性测试，并设计了两种此类测试，称为虚假日期测试。这些测试旨在检测LLM样本内预测中的两种偏见：先行偏见和上下文偏见。根据实证结果，本研究中测试的现代LLM中没有一个通过了第一个测试，表明它们的样本内预测存在先行偏见。
英文摘要：
Large language models (LLMs) are a type of machine learning tool that economists have started to apply in their empirical research. One such application is macroeconomic forecasting with backtesting of LLMs, even though they are trained on the same data that is used to estimate their forecasting performance. Can these in-sample accuracy results be extrapolated to the model's out-of-sample performance? To answer this question, we developed a family of prompt sensitivity tests and two members of this family, which we call the fake date tests. These tests aim to detect two types of biases in LLMs' in-sample forecasts: lookahead bias and context bias. According to the empirical results, none of the modern LLMs tested in this study passed our first test, signaling the presence of lookahead bias in their in-sample forecasts.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
