发表机构
Peking University; University of Central Florida（北京大学; 中佛罗里达大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
研究发现，攻击者仅通过发布新闻文章即可系统操纵LLM概率预测，单篇文章使56%预测翻转，且现有防御均可被廉价绕过，揭示了LLM预测对信息供应链的脆弱性。
AI 中文摘要
大型语言模型（LLMs）越来越多地通过检索和推理新闻来预测现实世界事件。我们表明，这种对开放、可爬取的新闻语料库的依赖创造了一个新的攻击面：一个仅能发布文章的攻击者——无需访问检索器、模型或用户查询——就能系统地移动预测器的输出概率。我们形式化了概率预测器的新闻语料库投毒，这是一种不同于先前RAG投毒的威胁模型，后者针对事实答案或观点极性而非校准概率。我们在500个已解决的ForecastBench问题上评估了该攻击，使用具有严格爬取日期截止的1740万篇文章的Common Crawl News语料库，基于三个构建在开放7-8B模型上的检索增强预测器。每个问题单篇LLM撰写的文章使56%的预测跨越0.5边界翻转；五篇文章使69-73%的预测翻转，并将概率净移动+0.13至+0.22（相对于中性文章安慰剂），使Brier分数从0.18恶化到0.37。该效应在注入文章的数量、检索排名、查询相似性和上下文占比上是单调的，跨模型家族迁移，且不受声称的发布者影响。然后我们评估了三种自然防御——来源白名单、隔离后聚合预测和困惑度过滤——并显示每种都有廉价绕过方式：分别通过伪造发布者、多数投毒和更高温度生成。我们的结果表明，概率性LLM判断继承了它们所消费的信息供应链的全部脆弱性。
英文摘要
Large language models (LLMs) are increasingly used to forecast real-world events by retrieving and reasoning over news. We show that this dependence on an open, crawlable news corpus creates a new attack surface: an adversary who can merely publish articles--without access to the retriever, the model, or the user's queries--can systematically move the forecaster's output probabilities. We formalize news-corpus poisoning of probabilistic forecasters, a threat model distinct from prior RAG poisoning, which targets factual answers or opinion polarity rather than calibrated probabilities. We evaluate the attack on 500 resolved ForecastBench questions against a 17.4M-article Common Crawl News corpus with a strict crawl-date cutoff, using three retrieval-augmented forecasters built on open 7-8B models. A single LLM-written article per question flips 56% of forecasts across the 0.5 boundary; five articles flip 69-73% and shift probabilities by +0.13 to +0.22 net of a neutral-article placebo, degrading the Brier score from 0.18 to 0.37. The effect is monotone in the number, retrieval rank, query similarity, and context share of injected articles, transfers across model families, and is unaffected by the claimed publisher. We then evaluate three natural defenses--source allow-lists, isolate-then-aggregate forecasting, and perplexity filtering--and show that each has a cheap bypass: spoofed publishers, majority poisoning, and higher-temperature generation, respectively. Our results indicate that probabilistic LLM judgments inherit the full fragility of the information supply chain they consume.
CommentsKey words: LLM forecasting, retrieval-augmented generation, data poisoning, misinformation, calibration
