OmniClimate-TC：面向热带气旋多媒体推理的物理感知视觉抽象
OmniClimate-TC: Physics-Aware Visual Abstractions for Multimedia Reasoning over Tropical Cyclones
- National University of Singapore（新加坡国立大学）
- Nanyang Technological University（南洋理工大学）
机构由 AI 辅助整理，请以论文原文为准。
中文总结 AI 辅助
该研究针对VLMs处理热带气旋灾害场时的感知不匹配问题，提出PAVA接口并构建OmniClimate-TC基准，证实该表示设计可提升VLMs的相关推理能力。
中文摘要 AI 辅助
气象再分析通过连续的、受物理约束的场编码极端天气，这对视觉语言模型（VLMs）构成了根本性挑战，因为VLMs的感知假设由自然图像塑造。热带气旋是这种不匹配的典型例子：强度极值、不对称性、空间范围和物理影响等关键属性源于场级组织，而非以对象为中心的视觉线索。现有方法通过文本对齐或注释来解决这一差距，将该问题视为多模态监督而非表示设计。我们引入物理感知视觉抽象（PAVA），这是一种即插即用的物理感知表示与注释接口，可将物理再分析场映射为视觉可识别且语义有依据的感知抽象，用于视觉语言推理中的监督与评估。基于PAVA，我们构建了OmniClimate-TC，这是一个涵盖五类推理和九项任务的热带气旋分析基准，包含243890个基于物理的指令调优对。利用PAVA对齐的监督，我们对VLMs进行适配，并提供证据表明这种表示设计可改善对热带气旋灾害场的推理。我们的结果表明，OmniClimate-TC是面向结构化地球物理场多媒体推理的基准，并强调表示设计是科学媒体中基于物理的推理的关键要素。
英文摘要
Meteorological reanalysis encodes extreme weather through continuous, physically constrained fields, posing a fundamental challenge for vision-language models (VLMs) whose perceptual assumptions are shaped by natural images. Tropical cyclones exemplify this mismatch: critical properties such as intensity extrema, asymmetry, spatial extent, and physical impacts arise from field-level organization rather than object-centric visual cues. Existing approaches address this gap through text alignment or annotation, treating the problem as multimodal supervision rather than representation design. We introduce Physics-Aware Visual Abstraction (PAVA), a plug-and-play physics-aware representation and annotation interface that maps physical reanalysis fields to visually identifiable and semantically grounded perceptual abstractions for supervision and evaluation in vision-language reasoning. Building on PAVA, we construct OmniClimate-TC, a benchmark for tropical cyclone analysis spanning five classes of reasoning and nine tasks, with 243,890 physically grounded instruction-tuning pairs. Using PAVA-aligned supervision, we adapt VLMs and provide evidence that this representation design improves reasoning over tropical cyclone hazard fields. Our results position OmniClimate-TC as a benchmark for multimedia reasoning over structured geophysical fields, and highlight representation design as a key ingredient for physically grounded reasoning in scientific media.
