Context-Aware Multimodal Representation Learning for Spatio-Temporally Explicit Environmental Modelling
Environmental Data Science and Remote Sensing Group(环境数据科学与遥感小组);Institute for Earth System Science and Remote Sensing(地球系统科学与遥感研究所);Leipzig University(莱比锡大学);German Centre for Integrative Biodiversity Research (iDiv)(整合生物多样性研究德国中心(iDiv))
Julia Peters, Karin Mora, Miguel D. Mahecha, Chaonan Ji, David Montero, Clemens Mosig, Guido Kraemer
备注:
10 pages (incliding 2 pages of references), 7 figures
Earth observation (EO) foundation models have emerged as an effective approach to derive latent representations of the Earth system from various remote sensing sensors. These models produce embeddings that can be used as analysis-ready datasets, enabling the modelling of ecosystem dynamics without extensive sensor-specific preprocessing. However, existing models typically operate at fixed spatial or temporal scales, limiting their use for ecological analyses that require both fine spatial detail and high temporal fidelity. To overcome these limitations, we propose a representation learning framework that integrates different EO modalities into a unified feature space at high spatio-temporal resolution. We introduce the framework using Sentinel-1 and Sentinel-2 data as representative modalities. Our approach produces a latent space at native 10 m resolution and the temporal frequency of cloud-free Sentinel-2 acquisitions. Each sensor is first modeled independently to capture its sensor-specific characteristics. Their representations are then combined into a shared model. This two-stage design enables modality-specific optimisation and easy extension to new sensors, retaining pretrained encoders while retraining only fusion layers. This enables the model to capture complementary remote sensing data and to preserve coherence across space and time. Qualitative analyses reveal that the learned embeddings exhibit high spatial and semantic consistency across heterogeneous landscapes. Quantitative evaluation in modelling Gross Primary Production reveals that they encode ecologically meaningful patterns and retain sufficient temporal fidelity to support fine-scale analyses. Overall, the proposed framework provides a flexible, analysis-ready representation learning approach for environmental applications requiring diverse spatial and temporal resolutions.
Decoupling Time and Space: A Temporally Conditioned Refinement for EEG Source Imaging
Berlin Institute for the Foundations of Learning and Data(柏林学习与数据基础研究所);Technische Universität Berlin(柏林工业大学);Korea University(高丽大学);Max Planck Institut für Informatik(马克斯·普朗克信息学研究所);Basque Center on Cognition, Brain and Language(巴斯克认知、大脑与语言中心);Ikerbasque(伊克尔巴斯克基金会)
Orca: Ocean Significant Wave Height Estimation with Spatio-temporally Aware Large Language Models
School of Data Science and Engineering, East China Normal University(华东师范大学数据科学与工程学院);The State Key Laboratory of Estuarine and Coastal Research, ECNU(华东师范大学河口海岸学国家重点实验室)
Advancing Spatiotemporal Prediction using Artificial Intelligence: Extending the Framework of Geographically and Temporally Weighted Neural Network (GTWNN) for Differing Geographical and Temporal Contexts
University of New South Wales(新南威尔士大学);City Futures Research Centre(城市未来研究中心)
SERT: A Transfomer Based Model for Spatio-Temporal Sensor Data with Missing Values for Environmental Monitoring
Hamilton Institute(汉密尔顿研究所);Insight Centre for Data Analytics(洞察数据分析中心);Maynooth University(梅努斯大学);University of León(莱昂大学);Dublin City University(都柏林城市大学)
Every Client Is an Environment: Federated De-confounding for Spatio-Temporal Forecasting
The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州));Shanghai Jiao Tong University(上海交通大学);Harbin Engineering University(哈尔滨工程大学)
CC-GRMAS: A Multi-Agent Graph Neural System for Spatiotemporal Landslide Risk Assessment in High Mountain Asia
Department of Computer Engineering(计算机工程系);Dwarkadas Jivanlal Sanghvi College of Engineering(德瓦尔卡斯·吉文拉尔·桑格维学院);College of Computing(计算机学院);Georgia Institute of Technology(佐治亚理工学院);Geo-Hydro Meteorological Risks Management Division(地质-水文气象灾害管理部);National Institute of Disaster Management(国家灾害管理研究所)
DyneTrion: A Spatio-temporally Coherent Generative Emulator for Protein Dynamics Across Timescales
Artificial Intelligence Innovation and Incubation Institute, Fudan University(复旦大学人工智能创新与孵化院);Shanghai Academy of AI for Science(上海人工智能科学研究院);School of Data Science, Fudan University(复旦大学数据科学学院);Institute for Big Data, Fudan University(复旦大学大数据研究院);MOE Laboratory for National Development and Intelligent Governance, Fudan University(复旦大学教育部国家发展与智能治理实验室)
Predicting and Interpolating Spatiotemporal Environmental Data: A Case Study of Groundwater Storage in Bangladesh
Department of Computer Science, Brunel University of London, UK(布鲁内尔大学伦敦计算机科学系);Department of Geography, UCL, London, UK(伦敦大学学院地理系);Department of Risk and Disaster Reduction, UCL, London, UK(伦敦大学学院风险与灾害减少系)