分析上下文在大语言模型预测中的作用
Analyzing the Role of Context in Forecasting with Large Language Models
AI总结：
本研究构建含600余个二元预测问题及配套新闻的数据集，探究不同上下文提示对大语言模型预测性能的影响，发现加入新闻可提升效果、少样本会降低准确率，且大模型表现更优，展现其在自动化预测中的潜力。
摘要：
本研究评估了近期语言模型（LLMs）在二元预测问题上的预测性能。我们首先构建了一个包含600余个二元预测问题的新型数据集，该数据集补充了相关新闻文章及其与问题相关的简洁摘要。随后我们探究了不同上下文层级的输入提示对预测性能的影响。结果表明，纳入新闻文章可显著提升性能，而使用少样本示例则会导致准确率下降。我们发现更大的模型始终优于更小的模型，凸显了LLMs在提升自动化预测方面的潜力。
英文摘要：
This study evaluates the forecasting performance of recent language models (LLMs) on binary forecasting questions. We first introduce a novel dataset of over 600 binary forecasting questions, augmented with related news articles and their concise question-related summaries. We then explore the impact of input prompts with varying level of context on forecasting performance. The results indicate that incorporating news articles significantly improves performance, while using few-shot examples leads to a decline in accuracy. We find that larger models consistently outperform smaller models, highlighting the potential of LLMs in enhancing automated forecasting.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
