WeatherFormer：一种用于从小数据集学习稳健天气表示的预训练编码器模型
WeatherFormer: A Pretrained Encoder Model for Learning Robust Weather Representations from Small Datasets
AI总结：
WeatherFormer是一种基于Transformer编码器的预训练模型，通过独特的时空编码和稳健预训练策略，从少量观测中学习天气特征，在县级大豆产量预测和流感预测中达到最先进性能。
摘要：
本文介绍了WeatherFormer，一种基于Transformer编码器的模型，旨在从极少量观测中学习稳健的天气特征。它解决了从小数据集建模复杂天气动态的挑战，这是农业、流行病学和气候科学中许多预测任务的瓶颈。WeatherFormer在一个大型预训练数据集上进行了预训练，该数据集包含美洲地区39年的卫星测量数据。通过新颖的预训练任务和微调，WeatherFormer在县级大豆产量预测和流感预测方面达到了最先进的性能。技术创新包括一种独特的时空编码，能够捕捉地理、年度和季节变化，将Transformer架构适应于连续天气数据，以及一种预训练策略，以学习对缺失天气特征具有稳健性的表示。本文首次证明了在大规模Transformer编码器模型上进行预训练对于跨多个领域的天气依赖应用的有效性。
英文摘要：
This paper introduces WeatherFormer, a transformer encoder-based model designed to learn robust weather features from minimal observations. It addresses the challenge of modeling complex weather dynamics from small datasets, a bottleneck for many prediction tasks in agriculture, epidemiology, and climate science. WeatherFormer was pretrained on a large pretraining dataset comprised of 39 years of satellite measurements across the Americas. With a novel pretraining task and fine-tuning, WeatherFormer achieves state-of-the-art performance in county-level soybean yield prediction and influenza forecasting. Technical innovations include a unique spatiotemporal encoding that captures geographical, annual, and seasonal variations, adapting the transformer architecture to continuous weather data, and a pretraining strategy to learn representations that are robust to missing weather features. This paper for the first time demonstrates the effectiveness of pretraining large transformer encoder models for weather-dependent applications across multiple domains.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
