ClimateIQA: 一种新的数据集和基准,以推进气象异常分析中的视觉-语言模型
ClimateIQA: A New Dataset and Benchmark to Advance Vision-Language Models in Meteorology Anomalies Analysis
AI总结:
ClimateIQA通过引入SPOT算法和新型数据集,提升视觉-语言模型在气象异常分析中的准确性和表现。
摘要:
气象热力图在解码极端天气现象中起着至关重要的作用,但其固有的复杂性,如不规则轮廓、无结构模式和复杂的颜色变化,为最先进的视觉-语言模型(VLMs)带来了独特的分析难题。当前最先进的模型,如GPT-4o、Qwen-VL和LLaVA 1.6,在精确的颜色识别和空间定位任务上表现不佳,导致解释不准确或不完整。为了解决这些挑战,我们引入了稀疏位置和轮廓跟踪(SPOT),一种专门设计用于处理视觉数据中不规则形状着色区域的新型算法。SPOT通过提取其空间坐标来识别和定位这些区域,使不规则形状能够获得结构化的表示。基于SPOT,我们构建了ClimateIQA,一个新型的气象视觉问答(VQA)数据集,包含26,280张高分辨率热力图和762,120条用于风速、总降水量、风寒指数和热量指数分析的指令样本。ClimateIQA通过整合空间线索、地理元数据和再分析数据来增强VLM训练,提高模型在解释和描述极端天气特征方面的准确性。此外,我们开发了Climate-Zoo,一套基于SPOT赋能的ClimateIQA微调的VLMs,显著优于现有模型在气象热力图任务中的表现。
英文摘要:
Meteorological heatmaps play a vital role in deciphering extreme weather phenomena, yet their inherent complexities marked by irregular contours, unstructured patterns, and complex color variations present unique analytical hurdles for state-of-the-art Vision-Language Models (VLMs). Current state-of-the-art models like GPT-4o, Qwen-VL, and LLaVA 1.6 struggle with tasks such as precise color identification and spatial localization, resulting in inaccurate or incomplete interpretations. To address these challenges, we introduce Sparse Position and Outline Tracking (SPOT), a novel algorithm specifically designed to process irregularly shaped colored regions in visual data. SPOT identifies and localizes these regions by extracting their spatial coordinates, enabling structured representations of irregular shapes. Building on SPOT, we construct ClimateIQA, a novel meteorological visual question answering (VQA) dataset, comprising 26,280 high-resolution heatmaps and 762,120 instruction samples for wind gust, total precipitation, wind chill index and heat index analysis. ClimateIQA enhances VLM training by incorporating spatial cues, geographic metadata, and reanalysis data, improving model accuracy in interpreting and describing extreme weather features. Furthermore, we develop Climate-Zoo, a suite of fine-tuned VLMs based on SPOT-empowered ClimateIQA, which significantly outperforms existing models in meteorological heatmap tasks.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。