arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2607.02829cs.LG
收藏

更少令牌,更优预测:用于高效天气预报的稀疏残差路由

Less Tokens, Better Forecasts: Sparse Residual Routing for Efficient Weather Prediction

Tulane University(杜兰大学);Northeastern University(东北大学);University of Alabama at Birmingham(阿拉巴马大学伯明翰分校);Oak Ridge National Laboratory(橡树岭国家实验室)

Janet Wang, Yunbei Zhang, Lin Zhao, Xi Xiao, Jihun Hamm, Xiao Wang

AI总结:

研究针对现有基于ViT的天气预报模型计算冗余问题,提出无参数插件路由模块Sparse-Reslim,通过稀疏令牌处理兼顾固定网格要求,提升预报精度并大幅降低成本。

摘要:

现有的基于ViT的天气预报模型在所有空间令牌上进行统一计算,导致大量中间计算冗余。标准的令牌效率方法不适用于天气预报这种时空密集预测问题。我们引入了Sparse-Reslim,一种无参数插件路由模块,使稀疏令牌处理与固定网格要求兼容,提高了预测精度并降低了成本。

英文摘要:

Existing ViT-based weather forecasting models apply uniform computation across all spatial tokens, even though nearby atmospheric grid points often contain similar values and large regions evolve smoothly over time. This makes much of the intermediate per-token computation redundant. Standard token-efficiency methods, such as pruning or merging, reduce cost by removing or fusing tokens. However, weather forecasting is a spatiotemporal dense prediction problem in which a history of atmospheric states must be mapped to future values on the original latitude-longitude grid. Thus, every grid cell must retain a physically meaningful representation, especially under autoregressive rollout. We introduce Sparse-Reslim, a parameter-free plug-in routing module that makes sparse token processing compatible with this fixed-grid requirement. Sparse-Reslim routes only 25% of spatial tokens through the expensive middle transformer blocks and treats those blocks as residual updates: it computes the change produced for the routed tokens and scatters only this delta back to the full sequence. Unselected tokens keep their pre-routing representations exactly, so no grid cell is dropped or replaced by a mask token, and no fusion layer or additional parameters are introduced. Across ERA5 resolutions up to the operational 0.25\textdegree{} standard and two model families, a deterministic Transformer and a diffusion model, Sparse-Reslim improves forecast accuracy on every evaluated variable while substantially reducing cost: training is about 2.5x faster in the main settings and reaches 3.18x speedup at 0.25\textdegree{}, with over 2.2x lower peak memory. A controlled decomposition shows that the accuracy gain comes primarily from sparse routing itself, while random token selection provides an additional regularization benefit without selector overhead.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑