arXiv 2508.099042026-07-14cs.LGcs.AI

超越简单提示：改进LLMs上下文辅助预测的策略

Beyond Naïve Prompting: Strategies for Improved Context-aided Forecasting with LLMs

ServiceNow Research（ServiceNow研究院）；Mila - Québec AI Institute（Mila——魁北克人工智能研究所）；Université de Montréal（蒙特利尔大学）；McGill University（麦吉尔大学）；Polytechnique Montréal（蒙特利尔理工学院）；Université Laval（拉瓦尔大学）

Arjun Ashok, Andrew Robert Williams, Vincent Zhihao Zheng, Irina Rish, Nicolas Chapados, Étienne Marcotte, Valentina Zantedeschi, Alexandre Drouin

备注：: Published at TMLR - OpenReview link: https://openreview.net/forum?id=dkjHHFJkVI

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文提出四种策略，从模型诊断、准确性和效率三个正交维度改进LLMs的上下文辅助预测，通过广泛实验揭示执行差距、性能提升和成本降低的解决方案。

摘要：

现实中的预测要求模型不仅要整合历史数据，还要整合以文本形式提供的相关上下文信息。尽管大型语言模型（LLMs）在上下文辅助预测中展现出潜力，但仍存在关键挑战：我们缺乏诊断工具来理解失败模式，性能远低于其潜力，且高计算成本限制了实际部署。我们引入了一个统一的框架，包含四种策略，以解决这些限制，从三个正交维度：模型诊断、准确性和效率。通过在从小众开源模型到前沿模型（包括Gemini、GPT和Claude）的广泛评估中，我们揭示了基本见解和实用解决方案。我们的发现涵盖三个关键维度：诊断策略揭示了“执行差距”，即模型正确解释了上下文如何影响预测，但未能应用这种推理；以准确性为核心的策略实现了25-50%的显著性能提升；以效率为导向的方法显示，适应性路由在小型和大型模型之间可以平均接近大型模型的准确性，同时显著降低推理成本。这些正交策略可以根据部署限制灵活集成，为实践者提供了一套全面的LLM上下文辅助预测工具包。

英文摘要：

Real-world forecasting requires models to integrate not only historical data but also relevant contextual information provided in textual form. While large language models (LLMs) show promise for context-aided forecasting, critical challenges remain: we lack diagnostic tools to understand failure modes, performance remains far below their potential, and high computational costs limit practical deployment. We introduce a unified framework of four strategies that address these limitations along three orthogonal dimensions: model diagnostics, accuracy, and efficiency. Through extensive evaluation across model families from small open-source models to frontier models including Gemini, GPT, and Claude, we uncover both fundamental insights and practical solutions. Our findings span three key dimensions: diagnostic strategies reveal the "Execution Gap" where models correctly explain how context affects forecasts but fail to apply this reasoning; accuracy-focused strategies achieve substantial performance improvements of 25-50%; and efficiency-oriented approaches show that adaptive routing between small and large models can approach large model accuracy on average while significantly reducing inference costs. These orthogonal strategies can be flexibly integrated based on deployment constraints, providing practitioners with a comprehensive toolkit for practical LLM-based context-aided forecasting. Code is made available at https://github.com/ashok-arjun/beyond-naive-prompting.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。