arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2210.11269cs.LGphysics.ao-phphysics.flu-dyn
收藏

基于真实世界稀疏测量的推断

Inference from Real-World Sparse Measurements

Idiap Research Institute(伊迪亚普研究所);École Polytechnique Fédérale de Lausanne(洛桑联邦理工学院);Université de Genève(日内瓦大学)

Arnaud Pannatier, Kyle Matoba, François Fleuret

AI总结:

针对稀疏、不规则时空测量推断,本文提出无编码器-解码器、统一位置编码的 ViT 式注意力模型,在多领域优于现有 GNN 并显著降低 RMSE。

摘要:

现实世界问题通常涉及复杂且非结构化的测量集合,这类情况出现在传感器在空间或时间上稀疏布置时。能够对这种不规则时空数据进行建模并提取有意义的预报至关重要。能够处理位置随集合变化的测量集合并在任意位置提取读出值的深度学习架构,在方法上具有难度。当前最先进的模型是图神经网络,并且需要特定领域知识才能正确设置。 我们提出一种关注鲁棒性和实际适用性的基于注意力的模型,具有两项关键设计贡献。首先,我们采用类似 ViT 的 transformer,将上下文点和读出位置同时作为输入,消除了对编码器-解码器结构的需求。其次,我们使用统一方法对上下文位置和读出位置进行编码。该方法有意保持简单,并能与其他系统良好集成。与现有方法相比,我们的模型更简单、所需专业知识更少,并且不会受到有问题的瓶颈效应影响,所有这些都促成了更优性能。 我们进行了深入的消融研究,刻画了替代模型潜在表示中的这种有问题的瓶颈,它会抑制信息利用并阻碍训练效率。我们还在多个问题领域开展实验,包括高空风临近预报、两天天气预报、流体动力学和热扩散。在处理不规则采样数据时,我们的基于注意力的模型始终优于最先进模型。值得注意的是,我们的模型将风临近预报的均方根误差(RMSE)从 9.24 降至 7.98,并将热扩散任务的 RMSE 从 0.126 降至 0.084。

英文摘要:

Real-world problems often involve complex and unstructured sets of measurements, which occur when sensors are sparsely placed in either space or time. Being able to model this irregular spatiotemporal data and extract meaningful forecasts is crucial. Deep learning architectures capable of processing sets of measurements with positions varying from set to set, and extracting readouts anywhere are methodologically difficult. Current state-of-the-art models are graph neural networks and require domain-specific knowledge for proper setup. We propose an attention-based model focused on robustness and practical applicability, with two key design contributions. First, we adopt a ViT-like transformer that takes both context points and read-out positions as inputs, eliminating the need for an encoder-decoder structure. Second, we use a unified method for encoding both context and read-out positions. This approach is intentionally straightforward and integrates well with other systems. Compared to existing approaches, our model is simpler, requires less specialized knowledge, and does not suffer from a problematic bottleneck effect, all of which contribute to superior performance. We conduct in-depth ablation studies that characterize this problematic bottleneck in the latent representations of alternative models that inhibit information utilization and impede training efficiency. We also perform experiments across various problem domains, including high-altitude wind nowcasting, two-day weather forecasting, fluid dynamics, and heat diffusion. Our attention-based model consistently outperforms state-of-the-art models in handling irregularly sampled data. Notably, our model reduces the root mean square error (RMSE) for wind nowcasting from 9.24 to 7.98 and for heat diffusion tasks from 0.126 to 0.084.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑