Generating learning-friendly representations for points in a 2D space is a fundamental and long-standing problem in machine learning. Recently, multi-scale encoding schemes (such as Space2Vec) were proposed to directly encode any point in 2D space as a high-dimensional vector, and has been successfully applied to various (geo)spatial prediction tasks. However, a map projection distortion problem rises when applying location encoding models to large-scale real-world GPS coordinate datasets (e.g., species images taken all over the world) - all current location encoding models are designed for encoding points in a 2D (Euclidean) space but not on a spherical surface, e.g., earth surface. To solve this problem, we propose a multi-scale location encoding model called Sphere2V ec which directly encodes point coordinates on a spherical surface while avoiding the mapprojection distortion problem. We provide theoretical proof that the Sphere2Vec encoding preserves the spherical surface distance between any two points. We also developed a unified view of distance-reserving encoding on spheres based on the Double Fourier Sphere (DFS). We apply Sphere2V ec to the geo-aware image classification task. Our analysis shows that Sphere2V ec outperforms other 2D space location encoder models especially on the polar regions and data-sparse areas for image classification tasks because of its nature for spherical surface distance preservation.
3D Cloud reconstruction through geospatially-aware Masked Autoencoders
National Observatory of Athens（雅典国家天文台）；National Technical University of Athens（雅典国立技术大学）；Universitat de València（瓦伦西亚大学）；University of Oxford（牛津大学）；Royal Belgian Institute of Natural Sciences（比利时皇家自然科学研究所）；Imperial College London（伦敦帝国学院）；University of Sussex（萨塞克斯大学）；European Space Agency（欧洲空间局）；CSIC-UCM-IGEO（西班牙高等科研理事会-马德里康普顿斯大学-地球与环境研究所）
本研究结合 MSG/SEVIRI 图像与 CloudSat/CPR 雷达廓线，采用 MAE 及地理空间感知 SatMAE 进行自监督预训练和微调，实现优于 U-Nets 的 3D 云结构重建。
TrajGANR: Trajectory-Centric Urban Multimodal Learning via Geospatially Aligned Neural Representations
Google Research, Mountain View, CA（谷歌研究，山景城，加利福尼亚州）；Google LLC, Mountain View, CA（谷歌公司，山景城，加利福尼亚州）；Dept. of Computer Science, University of Southern California, Los Angeles, CA（计算机科学系，南加州大学，洛杉矶，加利福尼亚州）；SEAI Lab, Dept. of Geography and the Environment, The University of Texas at Austin, Austin, TX（SEAI实验室，地理与环境系，德克萨斯大学奥斯汀分校，奥斯汀，德克萨斯州）
Accurate and timely air quality and weather predictions are of great importance to urban governance and human livelihood. Though many efforts have been made for air quality or weather prediction, most of them simply employ one another as feature input, which ignores the inner-connection between two predictive tasks. On the one hand, the accurate prediction of one task can help improve another task's performance. On the other hand, geospatially distributed air quality and weather monitoring stations provide additional hints for city-wide spatiotemporal dependency modeling. Inspired by the above…
University of Electronic Science and Technology of China（电子科技大学）；Sichuan University（四川大学）；Eye See Inc.（Eye See公司）；Yunnan University（云南大学）
Background: Quantitative prediction of the adolescents' spherical equivalent based on their variable-length historical vision records. Methods: From October 2019 to March 2022, we examined binocular uncorrected visual acuity, axial length, corneal curvature, and axial of 75,172 eyes from 37,586 adolescents aged 6-20 years in Chengdu, China. 80\% samples consist of the training set and the remaining 20\% form the testing set. Time-Aware Long Short-Term Memory was used to quantitatively predict the adolescents' spherical equivalent within two and a half years. Result: The mean absolute predictio…
Automated Detection of Myopic Maculopathy in MMAC 2023: Achievements in Classification, Segmentation, and Spherical Equivalent Prediction
LaTIM UMR 1101, Inserm（法国国家健康与医学研究院LaTIM UMR 1101实验室）；Univ Bretagne Occidentale（西布列塔尼大学）；Evolucare Technologies（Evolucare科技公司）；University of Electronic Science and Technology of China（电子科技大学）；College of Computer Science, Sichuan University（四川大学计算机学院）；IMT Atlantique
Convolutional Neural Networks (CNN) possess many positive qualities when it comes to spatial raster data. Translation invariance enables CNNs to detect features regardless of their position in the scene. However, in some domains, like geospatial, not all locations are exactly equal. In this work, we propose localized convolutional neural networks that enable convolutional architectures to learn local features in addition to the global ones. We investigate their instantiations in the form of learnable inputs, local weights, and a more general form. They can be added to any convolutional layers,…
Spatially Aware Deep Learning for Microclimate Prediction from High-Resolution Geospatial Imagery
Tel Aviv University, Faculty of Life Sciences, School of Zoology（特拉维夫大学，生命科学学院，动物学系）；Tel Aviv University, Center for Artificial Intelligence & Data Science（特拉维夫大学，人工智能与数据科学中心）；School of BioSciences, The University of Melbourne（墨尔本大学生物科学学院）
CrossVIT-augmented Geospatial-Intelligence Visualization System for Tracking Economic Development Dynamics
Center for Applied Statistics（应用统计中心）；School of Statistics（统计学院）；Renmin University of China（中国人民大学）；School of Information（信息学院）；Department of Statistics and Applied Probability（统计与应用概率系）；University of California Santa Barbara（加州大学圣巴巴拉分校）
Building age estimation based on widely available multi-source1 geospatial data
The University of Hong Kong（香港大学）；Institute for Climate and Carbon Neutrality, the University of Hong Kong（香港大学气候与碳中和研究所）；The University of Hong Kong Shenzhen Institute of Research and Innovation（香港大学深圳研究院）；School of Information Engineering, Guangdong University of Technology（广东工业大学信息工程学院）；Institute for Smart Cities, School of Architecture and Urban Planning, Shenzhen University（深圳大学建筑与城市规划学院智慧城市研究院）；College of Urban and Environmental Sciences, Academy of Frontier Interdisciplinary Research, Central China Normal University（华中师范大学城市与环境科学学院前沿交叉研究院）；University of North Carolina（北卡罗来纳大学）；Chulalongkorn University（朱拉隆功大学）
Source-Free Domain Adaptation for Geospatial Point Cloud Semantic Segmentation
Aerospace Information Research Institute, Chinese Academy of Sciences（中国科学院航天信息研究所）；International Research Center of Big Data for Sustainable Development Goals（可持续发展目标大数据国际研究中心）；University of Chinese Academy of Sciences（中国科学院大学）；Zhengzhou Institute for Advanced Research of Henan Polytechnic University（河南理工大学郑州研究院）；Henan Polytechnic University（河南理工大学）；School of Aeronautic Engineering, Changsha University of Science and Technology（长沙理工大学航空工程学院）；China University of Geosciences, Beijing（中国地质大学（北京））
Multi-modal learning for geospatial vegetation forecasting
Max-Planck-Institute for Biogeochemistry（马克斯·普朗克生物地球化学研究所）；ELLIS Unit Jena（耶拿ELLIS单元）；ETH Zürich（苏黎世联邦理工学院）；Hong Kong University of Science and Technology（香港科技大学）
OSMGraphCLIP: Learning Global Location Representations from OpenStreetMap Graphs
Harokopio University of Athens（雅典哈罗科皮奥大学）；National Technical University of Athens（雅典国家技术大学）；Vienna University of Technology（维也纳技术大学）；National Observatory of Athens（雅典国家天文台）
Improving the interpretability of geospatial artificial intelligence (GeoAI) models has become critically important to open the "black box" of complex AI models, such as deep learning. This paper compares popular saliency map generation techniques and their strengths and weaknesses in interpreting GeoAI and deep learning models' reasoning behaviors, particularly when applied to geospatial analysis and image processing tasks. We surveyed two broad classes of model explanation methods: perturbation-based and gradient-based methods. The former identifies important image areas, which help machines…
Macquarie University（麦考瑞大学）；Australian Institute of Health Innovation（澳大利亚健康创新研究院）
Pre-processing and Data Augmentation play an important role in Deep Convolutional Neural Networks (DCNN). Whereby several methods aim for standardization and augmentation of the dataset, we here propose a novel method aimed to feed DCNN with spherical space transformed input data that could better facilitate feature learning compared to standard Cartesian space images and volumes. In this work, the spherical coordinates transformation has been applied as a preprocessing method that, used in conjunction with normal MRI volumes, improves the accuracy of brain tumor segmentation and patient overa…
Stanford University（斯坦福大学）；Renmin University of China（中国人民大学）
Understanding intrinsic patterns and predicting spatiotemporal characteristics of cities require a comprehensive representation of urban neighborhoods. Existing works relied on either inter- or intra-region connectivities to generate neighborhood representations but failed to fully utilize the informative yet heterogeneous data within neighborhoods. In this work, we propose Urban2Vec, an unsupervised multi-modal framework which incorporates both street view imagery and point-of-interest (POI) data to learn neighborhood embeddings. Specifically, we use a convolutional neural network to extract …
School of Computer and Information, Hefei University of Technology（合肥工业大学计算机与信息学院）；School of Electrical Engineering and Automation, Hefei University of Technology（合肥工业大学电气与自动化工程学院）；The University of Electro-Communications,Japan（日本电气通信大学）
Field of view (FoV) prediction is critical in 360-degree video multicast, which is a key component of the emerging Virtual Reality (VR) and Augmented Reality (AR) applications. Most of the current prediction methods combining saliency detection and FoV information neither take into account that the distortion of projected 360-degree videos can invalidate the weight sharing of traditional convolutional networks, nor do they adequately consider the difficulty of obtaining complete multi-user FoV information, which degrades the prediction performance. This paper proposes a spherical convolution-e…
Recent microscopy imaging techniques allow to precisely analyze cell morphology in 3D image data. To process the vast amount of image data generated by current digitized imaging techniques, automated approaches are demanded more than ever. Segmentation approaches used for morphological analyses, however, are often prone to produce unnaturally shaped predictions, which in conclusion could lead to inaccurate experimental outcomes. In order to minimize further manual interaction, shape priors help to constrain the predictions to the set of natural variations. In this paper, we show how spherical …
Spherical Vision Transformers for Audio-Visual Saliency Prediction in 360-Degree Videos
Department of Computer Science and Engineering, Koç University（计算机科学与工程系，科克大学）；Department of Psychology, Boğaziçi University（心理学系，博多伊大学）；National Institute of Advanced Industrial Science and Technology (AIST), Intelligent Platforms Research Institute（国家先进工业科学与技术研究院（AIST），智能平台研究机构）；Department of Psychology, Boğaziçi University University（心理学系，博多伊大学）；Department of Computer Engineering, Hacettepe University（计算机工程系，哈切塞特佩大学）；KUIS AI Center（KUIS人工智能中心）
