发表机构
Western Kentucky University; University of Texas at Austin; RediMinds Inc.（西肯塔基大学; 德克萨斯大学奥斯汀分校; 睿智公司）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
针对LLMs生成高风险气象文本时虚构数值的问题，研究提出AFDBench基准，采用GRPO优化7B参数模型，提升其生成专业气象讨论的风格契合度与数据保真度。
AI 中文摘要
大型语言模型（LLMs）在生成高风险气象文本时会虚构数值，给气象传播带来风险。我们提出AFDBench，一款通过推理Google WeatherNext 2的结构化AI气象预报数据来生成专业区域预报讨论（AFDs）的AI气象学家。我们推出AFDBench，这是首个用于评估生成式气象推理的基准，包含来自13个国家气象局（NWS）站点的7732份专家撰写的讨论内容，以及真实的AI气象预报输入，还有三个互补指标：Met-Align（数值准确性）、Style-Align（专业术语契合度）、Input-Grounding（对源气象数据的保真度）。零样本评估显示，开源LLMs的Style-Align值较低（约0.33），Input-Grounding值中等（约0.88），无法以专业NWS语域写作或忠实地使用输入数据。我们应用针对温度准确性、天气学正确性和格式合规性的领域特定奖励的组相对策略优化（GRPO）。在来自两个未见过的NWS站点的1033个保留样本上，GRPO使Style-Align几乎翻倍，从0.318提升至0.619，Input-Grounding从0.881提升至0.940，证明强化学习能让70亿参数模型像专业气象学家一样写作并忠实地解释AI气象数据。
英文摘要
Large language models (LLMs) hallucinate numerical values when generating high-stakes meteorological text, posing risks for weather communication. We present AFDBench, an AI meteorologist that generates professional Area Forecast Discussions (AFDs) by reasoning through structured AI weather forecast data from Google's WeatherNext 2. We introduce AFDBench, the first benchmark for evaluating generative meteorological reasoning, comprising 7,732 expert written discussions from 13 National Weather Service (NWS) offices paired with real AI weather forecast inputs, and three complementary metrics: Met-Align (numerical accuracy), Style-Align (professional dialect adherence), and Input-Grounding (fidelity to source weather data). Zero-shot evaluations reveal that open-source LLMs achieve low Style-Align (~0.33) and moderate Input-Grounding (~0.88), failing to write in the professional NWS register or faithfully use their input data. We apply Group Relative Policy Optimization (GRPO) with domain-specific rewards targeting temperature accuracy, synoptic correctness, and format compliance. On 1,033 held-out samples from two unseen NWS offices, GRPO nearly doubles Style-Align from 0.318 to 0.619 and improves Input-Grounding from 0.881 to 0.940, demonstrating that reinforcement learning teaches a 7B-parameter model to write like a professional meteorologist and faithfully interpret AI weather data.
