The rapid growth of the satellite industry has driven a significant increase in geospatial data acquisition, highlighting a critical bottleneck: the severe disparity between the volume of collected sensor data and the limited downlink bandwidth available to ground stations. While On-Board Computing (OBC) has helped address this by pre-processing data in orbit, this article further advances the paradigm by introducing an in-sensor computing framework. We present an optimized end-to-end Earth Observation (EO) pipeline tailored for strict computational constraints by integrating TinyML techniques with the Sony IMX500 Intelligent Vision Sensor. Specifically, our approach shifts processing directly to the sensor level, offloading the computation from the primary embedded device, and effectively mitigating the downlink transmission of noisy or irrelevant data. We evaluated several efficient Convolutional Neural Networks (ConvNets), i.e., SqueezeNet, ShuffleNetV2, and MCUNetV1, on the EuroSAT dataset. Experimental results show that, despite the optimizations required for deployment on the IMX500 platform, our models maintain a competitive 96.68% accuracy while operating within its 8 MB constraints. Specifically, the models reach an average processing throughput of 17.40 FPS with a latency of 27.43 ms. Furthermore, our system profile exhibits high energy efficiency, with a low energy footprint of 14.19 mJ per inference and an efficiency rating of 42.26 GMAC/J, demonstrating its viability for in-sensor deployment.
BigEarthNet.txt: A Large-Scale Multi-Sensor Image-Text Dataset and Benchmark for Earth Observation
BIFOLD(柏林智能数据与机器学习研究所);Technische Universität Berlin(柏林工业大学);University of Trento(特伦托大学);National Technical University(雅典国家技术大学);National Observatory of Athens(雅典国家天文台)
Increasing the Robustness of Model Predictions to Missing Sensors in Earth Observation
University of Kaiserslautern-Landau, Kaiserslautern, Germany(凯撒斯劳滕-兰道大学,凯撒斯劳滕,德国);German Research Center for Artificial Intelligence, Kaiserslautern, Germany(德国人工智能研究中心,凯撒斯劳滕,德国)
Earth-OneVision: Extending Remote Sensing Multimodal Large Language Models to More Sensor Modalities and Tasks
National Key Laboratory of Science and Technology on Space-Born Intelligent Information Processing (SBIIP), Beijing Institute of Technology(北京理工大学空间智能信息处理国家重点实验室);Aerospace Information Research Institute, Chinese Academy of Sciences(中国科学院空天信息创新研究院);Key Laboratory of Technology in Geo-Spatial Information Processing and Application System, Chinese Academy of Sciences(中国科学院地理空间信息处理与应用系统技术重点实验室);Advanced Research Institute of Multidisciplinary Sciences, Beijing Institute of Technology(北京理工大学前沿交叉科学研究院);School of Mechatronical Engineering, Beijing Institute of Technology(北京理工大学机电学院);School of Earth and Space Sciences, Peking University(北京大学地球与空间科学学院);School of Electronics, Peking University(北京大学电子学院);School of Computer Science and Hubei Key Laboratory of Intelligent Geo-Information Processing(华中科技大学计算机科学与技术学院&湖北省智能地理信息处理重点实验室)
Earth-o1: A Grid-free Observation-native Atmospheric World Model
Shanghai Artificial Intelligence Laboratory(上海人工智能实验室);Department of Information Engineering(信息工程系);School of Electronic Information and Electrical Engineering(电子信息与电气工程学院);Department of Atmospheric and Oceanic Sciences(大气与海洋科学系);School of Information Science and Technology(信息科学与技术学院);State Key Laboratory of Earth System Numerical Modeling and Application, Institute of Atmospheric Physics(地球系统数值模拟与应用国家重点实验室,大气物理研究所);College of Computer Science and Artificial Intelligence(计算机科学与人工智能学院);Department of Earth and Environmental Engineering(地球与环境工程系);Chinese Academy of Meteorological Sciences(中国气象科学研究院);School of Atmospheric Sciences(大气科学学院)
Neural Plasticity-Inspired Multimodal Foundation Model for Earth Observation
Chair of Data Science in Earth Observation, Technical University of Munich (TUM)(地球观测数据科学教授职位,慕尼黑技术大学);Munich Center for Machine Learning(慕尼黑机器学习中心);AIML Lab, School of Computer Science, University of St. Gallen(人工智能实验室,圣加尔登大学计算机科学学院);School of Rural, Surveying and Geoinformatics Engineering, National Technical University of Athens(农村、测绘与地理信息工程学院,国家技术大学雅典);Image Processing Laboratory (IPL), Universitat de València(图像处理实验室(IPL),瓦伦西亚大学)
UniverSat: Resolution- and Modality-Agnostic Transformers for Earth Observation
LIGM, Ecole Nationale des Ponts et Chaussées, IP Paris, Univ Gustave Eiffel, CNRS(LIGM,巴黎国立桥梁与道路学院,IP巴黎,埃夫尔大学,CNRS);LASTIG, Univ Gustave Eiffel, IGN, ENSG(LASTIG,埃夫尔大学,国家地理信息学院,ENSG);IGN(国家地理信息学院);CNES(国家航天中心);EFEO(埃克塞特东方研究院)
Towards a Unified Copernicus Foundation Model for Earth Vision
Technical University of Munich(慕尼黑技术大学);Munich Center for Machine Learning(慕尼黑机器学习中心);National Technical University of Athens & National Observatory of Athens(雅典国家技术大学及雅典国家天文台);Harokopio University of Athens(雅典哈罗科波斯大学);NVIDIA(NVIDIA公司)
SpectralEarth-FM: Bringing Hyperspectral Imagery into Multimodal Earth Observation Pretraining
Chair of Data Science in Earth Observation(地球观测数据科学主任);Technical University of Munich(慕尼黑技术大学);Remote Sensing Technology Institute(遥感技术研究所);German Aerospace Center (DLR)(德国航空航天中心);Department of Aerospace Engineering(航空航天工程系);University of the Bundeswehr Munich(联邦国防军慕尼黑大学);LEAP;Munich Center for Machine Learning (MCML)(慕尼黑机器学习中心);Univ. Grenoble Alpes(格勒诺布尔阿尔卑斯大学);Inria(法国国家信息与自动化技术研究院);CNRS(法国国家科学研究中心);Grenoble INP(格勒诺布尔INP);LJK
EBBINNOT: A Hardware Efficient Hybrid Event-Frame Tracker for Stationary Dynamic Vision Sensors
Nanyang Technological University(南洋理工大学);University of California, Los Angeles(加州大学洛杉矶分校);National University of Singapore(新加坡国立大学);City University of Hong Kong(香港城市大学)
Multi-modal Co-learning for Earth Observation: Enhancing single-modality models via modality collaboration
Department of Computer Science, University of Kaiserslautern-Landau (RPTU)(科斯拉尔特伦大学计算机科学系);SDS, German Research Center for Artificial Intelligence (DFKI)(德国人工智能研究中心(DFKI));INRAE, UMR TETIS, University of Montpellier(蒙彼利埃大学UMR TETIS);CIRAD, UMR TETIS, University of Montpellier(蒙彼利埃大学UMR TETIS);INRIA, EVERGREEN, University of Montpellier(蒙彼利埃大学)