AsyncCouple-Flow：用于时空预测的异步跨模态耦合与流匹配
AsyncCouple-Flow: Asynchronous Cross-Modal Coupling and Flow Matching for Spatio-Temporal Forecasting
浏览论文内容
中文总结 AI 辅助
针对多模态时空预测中采样率不同、模态缺失和自回归误差累积问题，提出AsyncCouple-Flow，结合令牌稀疏化、异步耦合图和流匹配，在天气和交通基准上优于现有方法。
中文摘要 AI 辅助
多模态时空预测（MM-STF）通过结合物理场、卫星图像和原位传感器等异构数据源，支持天气临近预报、交通预测和地球系统建模。目前存在三个障碍：（i）不同模态具有不同的时空采样率，迫使进行有损插值以统一到同一网格；（ii）由于传感器故障或重访间隔，模态在部署时经常缺失，而大多数方法在完整可用性条件下训练；（iii）自回归解码器在长预测范围内累积误差，并因多模态条件而放大。我们提出AsyncCouple-Flow来联合解决这些问题。模态感知令牌稀疏化（MATS）模块执行尺度感知的令牌化，并使用共享的重要性评分器在每个时间步选择前k个令牌，生成等长序列。异步跨模态耦合图（ACCG）用可学习的图替代固定的交叉注意力，该图的边编码时间偏移、语义相似性和模态特定的物理先验，从而在任意异步和缺失情况下实现融合。流匹配预测头将多步预测建模为条件常微分方程，通过随机模态丢弃训练并联合积分，以避免自回归漂移。在ERA5+GOES+ISD天气预报和带多源辅助信息的PEMS-BAY交通预测上的实验表明，AsyncCouple-Flow优于最先进的基线，并且在多达两个模态缺失的情况下保持鲁棒性。代码将在论文被接收后发布。
英文摘要
Multi-modal spatio-temporal forecasting (MM-STF) supports weather nowcasting, traffic prediction, and earth-system modeling by combining heterogeneous sources such as physical fields, satellite imagery, and in-situ sensors. Three obstacles persist: (i) modalities have different spatio-temporal sampling rates, forcing lossy interpolation onto a unified grid; (ii) modalities are frequently missing at deployment due to sensor outages or revisit gaps, while most methods train with full availability; and (iii) autoregressive decoders accumulate errors over long horizons, amplified by multi-modal conditioning. We propose AsyncCouple-Flow to address these issues jointly. A Modality-Aware Token Sparsification (MATS) module performs scale-aware tokenization and uses a shared importance scorer to select top-k tokens per timestep, producing equal-length sequences. An Asynchronous Cross-Modal Coupling Graph (ACCG) replaces fixed cross-attention with a learnable graph whose edges encode time offsets, semantic similarity, and modality-specific physical priors, enabling fusion under arbitrary asynchrony and missingness. A Flow-Matching Forecasting Head models multi-step prediction as a conditional ODE, trained with stochastic modality dropout and integrated jointly to avoid autoregressive drift. Experiments on ERA5+GOES+ISD weather forecasting and PEMS-BAY traffic prediction with multi-source side information show that AsyncCouple-Flow outperforms state-of-the-art baselines and remains robust with up to two missing modalities. The code will be released upon acceptance.
发表机构
- Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）
- Emory University（埃默里大学）
- University of California, Berkeley（加州大学伯克利分校）
- Yale University（耶鲁大学）
- Stevens Institute of Technology（史蒂文斯理工学院）
- City University of New York（纽约城市大学）
机构由 AI 辅助整理，请以论文原文为准。
