Drone-based remote sensing combined with AI-driven methodologies has shown great potential for accurate mapping and monitoring of coral reef ecosystems. This study presents a novel multi-scale approach to coral reef monitoring, integrating fine-scale underwater imagery with medium-scale aerial imagery. Underwater images are captured using an Autonomous Surface Vehicle (ASV), while aerial images are acquired with an aerial drone. A transformer-based deep-learning model is trained on underwater images to detect the presence of 31 classes covering various coral morphotypes, associated fauna, and habitats. These predictions serve as annotations for training a second model applied to aerial images. The transfer of information across scales is achieved through a weighted footprint method that accounts for partial overlaps between underwater image footprints and aerial image tiles. The results show that the multi-scale methodology successfully extends fine-scale classification to larger reef areas, achieving a high degree of accuracy in predicting coral morphotypes and associated habitats. The method showed a strong alignment between underwater-derived annotations and ground truth data, reflected by an AUC (Area Under the Curve) score of 0.9251. This shows that the integration of underwater and aerial imagery, supported by deep-learning models, can facilitate scalable and accurate reef assessments. This study demonstrates the potential of combining multi-scale imaging and AI to facilitate the monitoring and conservation of coral reefs. Our approach leverages the strengths of underwater and aerial imagery, ensuring the precision of fine-scale analysis while extending it to cover a broader reef area.
The Coralscapes Dataset: Semantic Scene Understanding in Coral Reefs
Environmental Computational Science and Earth Observation Laboratory, École Polytechnique Fédérale de Lausanne, Switzerland（洛桑联邦理工学院环境计算科学与地球观测实验室）；Laboratory for Biological Geochemistry, École Polytechnique Fédérale de Lausanne, Switzerland（洛桑联邦理工学院生物地球化学实验室）；Centre for Ecology and Conservation, University of Exeter, United Kingdom（埃克塞特大学生态与保护中心）；School of the Environment, The University of Queensland, Australia（昆士兰大学环境学院）；Center for Advanced Surface Analysis, University of Lausanne, Switzerland（日内瓦大学高级表面分析中心）
AI-driven Dispensing of Coral Reseeding Devices for Broad-scale Restoration of the Great Barrier Reef
Queensland University of Technology（昆士兰理工大学）；QUT Centre for Robotics（昆士兰理工大学机器人中心）；School of Electrical Engineering and Robotics（电气工程与机器人学院）；Australian Institute of Marine Science（澳大利亚海洋科学研究所）
CoralVQA: A Large-Scale Visual Question Answering Dataset for Coral Reef Image Understanding
Beijing University of Posts and Telecommunications（北京邮电大学）；Technology Innovation Center for South China Sea Remote Sensing, Surveying and Mapping Collaborative Application, Ministry of Natural Resources（自然资源部南海遥感测绘协同应用技术创新中心）；South China Sea Development Research Institute, Ministry of Natural Resources（自然资源部南海发展研究 institute）；Inspur Computer Technology Co., Ltd（Inspur 计算机技术有限公司）；Shandong Key Laboratory of Advanced Computing（山东先进计算重点实验室）
Automatic Coral Detection with YOLO: A Deep Learning Approach for Efficient and Accurate Coral Reef Monitoring
Cadi Ayyad University（卡迪·阿雅德大学）；University of La Réunion（留尼汪大学）；IRD（法国国家可持续发展研究所）；IFREMER（法国海洋开发研究院）；CNRS（法国国家科学研究中心）；University of New Caledonia（新喀里多尼亚大学）；Institut de Recherche pour le Développement（法国国家可持续发展研究所）；MNHN（法国国家自然历史博物馆）；EPHE（巴黎高等研究实践学院）
CoralSCOP-LAT: Labeling and Analyzing Tool for Coral Reef Images with Dense Mask
Hong Kong University of Science and Technology（香港科技大学）；King Abdullah University of Science and Technology（阿卜杜拉国王科技大学）；University of Technology Sydney（悉尼科技大学）
Corals are the primary habitat-building life-form on reefs that support a quarter of the species in the ocean. A coral reef ecosystem usually consists of reefs, each of which is like a tall building in any city. These reef-building corals secrete hard calcareous exoskeletons that give them structural rigidity, and are also a prerequisite for our accurate 3D modeling and semantic mapping using advanced photogrammetric computer vision and machine learning. Underwater videography as a modern underwater remote sensing tool is a high-resolution coral habitat survey and mapping technique. In this pa…
CoralscapesV2: Panoptic and Fine-Grained Visual Scene Understanding in Coral Reefs
MIT（麻省理工学院）；EPFL（洛桑联邦理工学院）；University of Djibouti（吉布提大学）；Ministry of Environment and Sustainable Development of Djibouti（吉布提环境与可持续发展部）；Centre d’Études et de Recherche de Djibouti（吉布提研究与研究中心）；Red Sea University of Port Sudan（苏丹港红海大学）；Aqaba Marine Reserve, Jordan（约旦亚喀巴海洋保护区）；United Nations Development Programme Jordan（联合国开发计划署约旦办事处）；Mai Nefhi College of Science, Eritrea（厄立特里亚马伊内菲理学院）；Ministry of Marine Resources, Massawa, Eritrea（厄立特里亚马萨瓦海洋资源部）
CSIRO’s Data61（澳大利亚联邦科学与工业研究组织 Data61）；CSIRO Oceans & Atmosphere（澳大利亚联邦科学与工业研究组织海洋与大气研究所）；Queensland University of Technology（昆士兰科技大学）；Google（谷歌）
Crown-of-Thorn Starfish (COTS) outbreaks are a major cause of coral loss on the Great Barrier Reef (GBR) and substantial surveillance and control programs are underway in an attempt to manage COTS populations to ecologically sustainable levels. We release a large-scale, annotated underwater image dataset from a COTS outbreak area on the GBR, to encourage research on Machine Learning and AI-driven technologies to improve the detection, monitoring, and management of COTS populations at reef scale. The dataset is released and hosted in a Kaggle competition that challenges the international Machin…
Cutting-edge 3D reconstruction solutions for underwater coral reef images: A review and comparison
State Key Laboratory of Information Engineering in Surveying Mapping and Remote Sensing（信息工程测绘与遥感国家重点实验室）；Wuhan University（武汉大学）；Institute of Geodesy and Photogrammetry（大地测量与摄影测量研究所）；ETH Zurich（苏黎世联邦理工学院）；The Hong Kong Polytechnic University（香港理工大学）；Peking University（北京大学）
Multi-label classification for multi-temporal, multi-spatial coral reef condition monitoring using vision foundation model with adapter learning
Department of Socio-Cultural Environmental Studies, Graduate School of Frontier Sciences, The University of Tokyo（社会文化环境研究部门，前沿科学研究生院，东京大学）；Department of Environment Systems, Graduate School of Frontier Sciences, The University of Tokyo（环境系统部门，前沿科学研究生院，东京大学）；New Heaven Reef Conservation Program, Koh Tao, Surat Thani, 84360, Thailand（新天堂珊瑚礁保护计划， Koh Tao, Surat Thani, 84360, Thailand）；Data Science and AI Innovation Research Promotion Center, Shiga University（数据科学与人工智能创新研究促进中心，Shiga大学）；Interfaculty Initiative in Information Studies, The University of Tokyo（跨学际信息研究倡议，东京大学）
Behavior analysis of animals involves the observation of intraspecific and interspecific interactions among various organisms in the environment. Collective behavior such as herding in farm animals, flocking of birds, and shoaling and schooling of fish provide information on its benefits on collective survival, fitness, reproductive patterns, group decision-making, and effects in animal epidemiology. In marine ethology, the investigation of behavioral patterns in schooling species can provide supplemental information in the planning and management of marine resources. Currently, damselfish spe…
Depth-Resolved Coral Reef Thermal Fields from Satellite SST and Sparse In-Situ Loggers Using Physics-Informed Neural Networks
College of Science and Engineering, James Cook University（詹姆斯库克大学科学与工程学院）；Centre for AI and Data Science Innovation, James Cook University（詹姆斯库克大学人工智能与数据科学创新中心）
eStonefish-Scenes: A Sim-to-Real Validated and Robot-Centric Event-based Optical Flow Dataset for Underwater Vehicles
Robotics Research Institute (ViCOROB), Universitat de Girona (UdG), Spain 2 School of Engineering \& Physical Sciences, Heriot-Watt University, Edinburgh, UK
Centre Universitaire Condorcet（孔多塞大学中心）；University of Burgundy（勃艮第大学）；University of Girona（赫罗纳大学）；Heriot-Watt University（赫瑞瓦特大学）
Autonomous repair of deep-sea coral reefs is a recent proposed idea to support the oceans ecosystem in which is vital for commercial fishing, tourism and other species. This idea can be operated through using many small autonomous underwater vehicles (AUVs) and swarm intelligence techniques to locate and replace chunks of coral which have been broken off, thus enabling re-growth and maintaining the habitat. The aim of this project is developing machine vision algorithms to enable an underwater robot to locate a coral reef and a chunk of coral on the seabed and prompt the robot to pick it up. A…
CUREE: A Curious Underwater Robot for Ecosystem Exploration
Woods Hole Oceanographic Institution (WHOI)（伍兹霍尔海洋研究所）；MIT-WHOI Joint Program in Oceanography/Applied Ocean Science and Engineering（麻省理工学院-伍兹霍尔海洋研究所海洋学/应用海洋科学与工程联合项目）
YH-MINER: Multimodal Intelligent System for Natural Ecological Reef Metric Extraction
South China Sea Institute of Oceanology, Chinese Academy of Sciences（中国科学院南海海洋研究所）；National Astronomical Observatories of the Chinese Academy of Sciences（中国科学院国家天文台）；University of Chinese Academy of Sciences（中国科学院大学）；International Department of Affiliated High School of South China Normal University（华南师范大学附属中学国际部）；Alibaba Cloud（阿里云）
University of Granada（格拉纳达大学）；Technical University of Denmark (DTU)（丹麦技术大学）；Virginia Commonwealth University（弗吉尼亚联邦大学）
The recognition of coral species based on underwater texture images pose a significant difficulty for machine learning algorithms, due to the three following challenges embedded in the nature of this data: 1) datasets do not include information about the global structure of the coral; 2) several species of coral have very similar characteristics; and 3) defining the spatial borders between classes is difficult as many corals tend to appear together in groups. For this reason, the classification of coral species has always required an aid from a domain expert. The objective of this paper is to …
In this paper, we present the first large-scale dataset for semantic Segmentation of Underwater IMagery (SUIM). It contains over 1500 images with pixel annotations for eight object categories: fish (vertebrates), reefs (invertebrates), aquatic plants, wrecks/ruins, human divers, robots, and sea-floor. The images have been rigorously collected during oceanic explorations and human-robot collaborative experiments, and annotated by human participants. We also present a benchmark evaluation of state-of-the-art semantic segmentation approaches based on standard performance metrics. In addition, we …
Queensland University of Technology（昆士兰科技大学）；Data61, CSIRO（联邦科学与工业研究组织数据61研究所）
Monitoring coral reefs using underwater vehicles increases the range of marine surveys and availability of historical ecological data by collecting significant quantities of images. Analysis of this imagery can be automated using a model trained to perform semantic segmentation, however it is too costly and time-consuming to densely label images for training supervised models. In this letter, we leverage photo-quadrat imagery labeled by ecologists with sparse point labels. We propose a point label aware method for propagating labels within superpixel regions to obtain augmented ground truth fo…
Commonwealth Scientific and Industrial Research Organisation (CSIRO)（澳大利亚联邦科学与工业研究组织）；Australian National University（澳大利亚国立大学）
Underwater image restoration is of significant importance in unveiling the underwater world. Numerous techniques and algorithms have been developed in the past decades. However, due to fundamental difficulties associated with imaging/sensing, lighting, and refractive geometric distortions, in capturing clear underwater images, no comprehensive evaluations have been conducted of underwater image restoration. To address this gap, we have constructed a large-scale real underwater image dataset, dubbed `HICRD' (Heron Island Coral Reef Dataset), for the purpose of benchmarking existing methods and …
How many labels do you need? A decision framework for cross-habitat marine species recognition
College of Science and Engineering, James Cook University（詹姆斯·库克大学科学与工程学院）；Centre for AI and Data Science Innovation, James Cook University（詹姆斯·库克大学人工智能与数据科学创新中心）
