Foundation models have advanced machine learning across various modalities, including images. Recently multiple teams trained foundation models specialized for remote sensing applications. This line of research is motivated by the distinct characteristics of remote sensing imagery, specific applications and types of robustness useful for satellite image analysis. In this work we systematically challenge the idea that specific foundation models are more useful than general-purpose vision foundation models, at least in the small scale. First, we design a simple benchmark that measures generalization of remote sensing models towards images with lower resolution for two downstream tasks. Second, we train iBOT, a self-supervised vision encoder, on MillionAID, an ImageNet-scale satellite imagery dataset, with several modifications specific to remote sensing. We show that none of those pretrained models bring consistent improvements upon general-purpose baselines at the ViT-B scale.
Beyond Natural-Image Foundation Models: Benchmarking Satellite Pretraining for Ophthalmic Image Analysis
Faculty of Electrical Engineering and Computing, University of Zagreb(萨格勒布大学电气工程与计算机学院);University College London(伦敦大学学院);Institute of Ophthalmology, University College London(伦敦大学学院眼科研究所);Department of Computer Science, University College London(伦敦大学学院计算机科学系);NIHR Moorfields Biomedical Research Centre(NIHR穆尔菲尔德生物医学研究中心);Hawkes Institute, University College London(伦敦大学学院霍克斯研究所);Moorfields Eye Hospital NHS Foundation Trust(穆尔菲尔德眼科医院NHS基金会信托)
Contrastive Pretraining for Visual Concept Explanations of Socioeconomic Outcomes
Technical University of Munich(慕尼黑工业大学);Munich Center for Machine Learning(慕尼黑机器学习中心);Vrije Universiteit Amsterdam(阿姆斯特丹自由大学);Getulio Vargas Foundation(热图利奥·瓦加斯基金会);Inria(法国国家信息与自动化研究所);University of Montpellier(蒙彼利埃大学)
ADPretrain: Advancing Industrial Anomaly Detection via Anomaly Representation Pretraining
School of Information Science and Electronic Engineering, Shanghai Jiao Tong University(上海交通大学信息科学与电子工程学院);MoE Key Lab of Artificial Intelligence, AI Institute, Shanghai Jiao Tong University(上海交通大学人工智能MOE重点实验室);College of Artificial Intelligence, Nanjing Agricultural University(南京农业大学人工智能学院);Key Laboratory of Livestock Farming Equipment, Ministry of Agriculture and Rural Affairs, Nanjing Agricultural University(南京农业大学农业机械化关键实验室)
DIET-CP: Lightweight and Data Efficient Self Supervised Continued Pretraining
Fordham University(福特汉姆大学);Rutgers University(罗格斯大学);Pioneer Centre for AI(先锋人工智能中心);Cold Spring Harbor Laboratory(冷泉港实验室);Brown University(布朗大学);University of Copenhagen(哥本哈根大学)
Early Yield Prediction for Sugar Beet Fields using Satellite Data -- Learnings from Specialized Vision Transformers
Center for Artificial Intelligence and Robotics, Technical University of Applied Sciences Wuerzburg-Schweinfurt(人工智能与机器人中心,维尔茨堡-施韦因富特应用技术大学);Bielefeld University(比勒费尔德大学);green spin GmbH(绿旋有限公司)
Enhancement-Driven Pretraining for Robust Fingerprint Representation Learning
Centre for Visual Information Technology, International Institute of Information Technology, Hyderabad, India(视觉信息技术中心,国际信息学院,印度海得拉巴);Stony Brook University, USA(石溪大学)
CarbonCLIP: Enhance Carbon Prediction from Satellite Imagery via Integrated Street-View Semantics and Temporal Context Training
Energy Research Institute at NTU(国立理工学院能源研究所);Interdisciplinary Graduate Programme(跨学科研究生项目);School of Electrical and Electronic Engineering(电气电子工程学院);School of Public Administration and Policy(公共管理与政策学院);Asian School of the Environment(亚洲环境学院);Center for Climate Change and Environmental Health(气候变化与环境健康中心)
SatCLIP: Global, General-Purpose Location Embeddings with Satellite Imagery
Microsoft Research New England(微软研究院新英格兰分部);University of Colorado Boulder(科罗拉多大学博尔德分校);Harvard University(哈佛大学);Microsoft AI for Good Research Lab(微软AI公益研究实验室);Wageningen University(瓦赫宁根大学)