LEAF：一个用于事件增强预测的活体基准
LEAF: A Living Benchmark for Event-Augmented Forecasting
AI总结：
本文提出LEAF，首个用于事件增强预测的活体基准，通过递归检索代理系统和双代理交叉验证，提供全面相关文本辅助预测，评估LLM在复杂真实场景中的预测能力。
摘要：
大型语言模型（LLMs）越来越多地应用于预测。为了评估这一能力并缓解预训练数据污染，已提出几种活体基准。然而，现有基准要么因数据稀缺缺乏多维事件，要么聚焦于相对封闭环境。为评估LLM在复杂真实场景中的预测能力，我们提出LEAF，首个用于事件增强预测任务的活体基准，包括未来事件概率、趋势和时间序列预测。LEAF利用递归检索代理系统配以双代理交叉验证，提供全面相关辅助文本。评估最新专有和开源LLMs发现，这些模型能利用复杂事件提取的信号提升预测性能。在股票领域，发现LLM在自信识别为更可预测的股票上表现更好。此外，事件与目标股票呈现强相关性。为此，LEAF提供必要的动态更新测试环境，持续跟踪和推动事件驱动预测任务的进步。
英文摘要：
Large Language Models (LLMs) are increasingly applied to real-world forecasting tasks, yet evaluating their true predictive capability remains compromised by pre-training data contamination and look-ahead leakage in automated search. Existing benchmarks either rely on static contexts, restrict evaluations to narrow environments, or fail to audit auxiliary textual events for future information leakage. To establish a rigorous evaluation paradigm, we propose LEAF, the first living benchmark for event-augmented forecasting tasks, including trend, event, and time series forecasting. LEAF couples a recursive retrieval agent system with dual-agent cross-validation to gather comprehensive, relevant, and temporally aligned auxiliary context. A comprehensive audit across 500 tasks by 47 domain specialists demonstrates that our pipeline suppresses future information leakage from 8.6% to 1.6%. Across extensive evaluations of 16 frontier proprietary and open-weight models on our benchmark, we show that LLMs effectively extract signals from verified events to boost trend and event forecasting.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
