大型语言模型中的气候知识
Climate Knowledge in Large Language Models
AI总结：
本研究通过全球查询网格和ERA5验证，评估了大型语言模型回忆气候常态的能力，发现其能捕捉基本气候模式但存在空间误差，且难以再现长期温度变化的空间格局，并提出了可复现的评估基准。
摘要：
大型语言模型（LLMs）正越来越多地部署于气候相关应用中，在这些应用中，理解模型内部的气候学知识对于可靠性和错误信息风险评估至关重要。尽管采用日益广泛，但LLMs从参数化知识中回忆气候常态的能力在很大程度上仍未得到表征。我们研究了当代LLMs在无外部检索情况下回忆气候常态的能力，聚焦于一个典型查询：指定地点1991-2020年7月平均2米气温。我们构建了一个1°分辨率陆地点的全球查询网格，提供坐标和位置描述符，并将响应与ERA5再分析数据进行了验证。结果表明，LLMs编码了非平凡的气候结构，捕捉了纬度和地形模式，均方根误差为3-6°C，偏差为±1°C。然而，空间连贯的误差仍然存在，特别是在山区和高纬度地区。性能在海拔1500米以上急剧下降，均方根误差达到5-13°C，而低海拔地区为2-4°C。我们发现，包含地理背景（国家、城市、地区）平均可将误差减少27%，且较大的模型对位置描述符最为敏感。虽然模型捕捉了1950-1974年至2000-2024年间观测到的全球平均变暖幅度，但它们未能再现温度变化的空间模式，而这与评估气候变化直接相关。这一局限性凸显出，尽管LLMs可能捕捉当前气候分布，但它们难以表征对理解气候动态至关重要的长期温度变化的区域和局部表现。我们的评估框架为量化LLMs中的参数化气候知识提供了一个可复现的基准，并补充了现有的气候传播评估。
英文摘要：
Large language models (LLMs) are increasingly deployed for climate-related applications, where understanding internal climatological knowledge is crucial for reliability and misinformation risk assessment. Despite growing adoption, the capacity of LLMs to recall climate normals from parametric knowledge remains largely uncharacterized. We investigate the capacity of contemporary LLMs to recall climate normals without external retrieval, focusing on a prototypical query: mean July 2-m air temperature 1991-2020 at specified locations. We construct a global grid of queries at 1° resolution land points, providing coordinates and location descriptors, and validate responses against ERA5 reanalysis. Results show that LLMs encode non-trivial climate structure, capturing latitudinal and topographic patterns, with root-mean-square errors of 3-6 °C and biases of $\pm$1 °C. However, spatially coherent errors remain, particularly in mountains and high latitudes. Performance degrades sharply above 1500 m, where RMSE reaches 5-13 °C compared to 2-4 °C at lower elevations. We find that including geographic context (country, city, region) reduces errors by 27% on average, with larger models being most sensitive to location descriptors. While models capture the global mean magnitude of observed warming between 1950-1974 and 2000-2024, they fail to reproduce spatial patterns of temperature change, which directly relate to assessing climate change. This limitation highlights that while LLMs may capture present-day climate distributions, they struggle to represent the regional and local expression of long-term shifts in temperature essential for understanding climate dynamics. Our evaluation framework provides a reproducible benchmark for quantifying parametric climate knowledge in LLMs and complements existing climate communication assessments.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
