Computer Vision Tool for Detection, Mapping and Fault Classification of PV Modules in Aerial IR Videos
Forschungszentrum Jülich GmbH（于利希研究中心）；Helmholtz-Institute Erlangen-Nuremberg for Renewable Energies (HI ERN)（埃尔朗根-纽伦堡亥姆霍兹可再生能源研究所）；Institute Materials for Electronics and Energy Technology, Universität Erlangen-Nürnberg (FAU)（埃尔朗根-纽伦堡大学电子与能源技术材料研究所）
Lukas Bommes, Tobias Pickel, Claudia Buerhop-Lutz, Jens Hauch, Christoph Brabec, Ian Marius Peters
Increasing deployment of photovoltaics (PV) plants demands for cheap and fast inspection. A viable tool for this task is thermographic imaging by unmanned aerial vehicles (UAV). In this work, we develop a computer vision tool for the semi-automatic extraction of PV modules from thermographic UAV videos. We use it to curate a dataset containing 4.3 million IR images of 107842 PV modules from thermographic videos of seven different PV plants. To demonstrate its use for automated PV plant inspection, we train a ResNet-50 to classify ten common module anomalies with more than 90 % test accuracy. Experiments show that our tool generalizes well to different PV plants. It successfully extracts PV modules from 512 out of 561 plant rows. Failures are mostly due to an inappropriate UAV trajectory and erroneous module segmentation. Including all manual steps our tool enables inspection of 3.5 MW p to 9 MW p of PV installations per day, potentially scaling to multi-gigawatt plants due to its parallel nature. While we present an effective method for automated PV plant inspection, we are also confident that our approach helps to meet the growing demand for large thermographic datasets for machine learning tasks, such as power prediction or unsupervised defect identification.
Deep learning is revolutionizing the mapping industry. Under lightweight human curation, computer has generated almost half of the roads in Thailand on OpenStreetMap (OSM) using high-resolution aerial imagery. Bing maps are displaying 125 million computer-generated building polygons in the U.S. While tremendously more efficient than manual mapping, one cannot map out everything from the air. Especially for roads, a small prediction gap by image occlusion renders the entire road useless for routing. Misconnections can be more dangerous. Therefore computer-based mapping often requires local veri…
Mines Paris - PSL University（巴黎国立高等矿业学校 - PSL大学）；LuciSun（LuciSun公司）；RTE France（法国输电网络公司）；BDPV（BDPV机构）
Photovoltaic (PV) energy generation plays a crucial role in the energy transition. Small-scale PV installations are deployed at an unprecedented pace, and their integration into the grid can be challenging since public authorities often lack quality data about them. Overhead imagery is increasingly used to improve the knowledge of residential PV installations with machine learning models capable of automatically mapping these installations. However, these models cannot be easily transferred from one region or data source to another due to differences in image acquisition. To address this issue…
Microsoft AI for Good Research Lab（微软人工智能公益研究实验室）；Stanford RegLab（斯坦福监管实验室）
Concentrated Animal Feeding Operations (CAFOs) pose serious risks to air, water, and public health, but have proven to be challenging to regulate. The U.S. Government Accountability Office notes that a basic challenge is the lack of comprehensive location information on CAFOs. We use the USDA's National Agricultural Imagery Program (NAIP) 1m/pixel aerial imagery to detect poultry CAFOs across the continental United States. We train convolutional neural network (CNN) models to identify individual poultry barns and apply the best performing model to over 42 TB of imagery to create the first nati…
Wuhan University（武汉大学）；SenseTime Research（商汤科技研究院）；University of Electronic Science and Technology of China（电子科技大学）；Sun Yat-Sen University（中山大学）；CUHK-SenseTime joint Lab（香港中文大学-商汤科技联合实验室）
Extracting building footprints from aerial images is essential for precise urban mapping with photogrammetric computer vision technologies. Existing approaches mainly assume that the roof and footprint of a building are well overlapped, which may not hold in off-nadir aerial images as there is often a big offset between them. In this paper, we propose an offset vector learning scheme, which turns the building footprint extraction problem in off-nadir images into an instance-level joint prediction problem of the building roof and its corresponding "roof to footprint" offset vector. Thus the foo…
Linux Polska（波兰Linux公司）；University of Warsaw（华沙大学）；Agency for Restructuring and Modernisation of Agriculture（农业结构调整与现代化署）
Monitoring of land cover and land use is crucial in natural resources management. Automatic visual mapping can carry enormous economic value for agriculture, forestry, or public administration. Satellite or aerial images combined with computer vision and deep learning enable precise assessment and can significantly speed up change detection. Aerial imagery usually provides images with much higher pixel resolution than satellite data allowing more detailed mapping. However, there is still a lack of aerial datasets made for the segmentation, covering rural areas with a resolution of tens centime…
This paper presents OpenREALM, a real-time mapping framework for Unmanned Aerial Vehicles (UAVs). A camera attached to the onboard computer of a moving UAV is utilized to acquire high resolution image mosaics of a targeted area of interest. Different modes of operation allow OpenREALM to perform simple stitching assuming an approximate plane ground, or to fully recover complex 3D surface information to extract both elevation maps and geometrically corrected orthophotos. Additionally, the global position of the UAV is used to georeference the data. In all modes incremental progress of the resul…
With the increase in use of Unmanned Aerial Vehicles (UAVs)/drones, it is important to detect and identify causes of failure in real time for proper recovery from a potential crash-like scenario or post incident forensics analysis. The cause of crash could be either a fault in the sensor/actuator system, a physical damage/attack, or a cyber attack on the drone's software. In this paper, we propose novel architectures based on deep Convolutional and Long Short-Term Memory Neural Networks (CNNs and LSTMs) to detect (via Autoencoder) and classify drone mis-operations based on sensor data. The pro…
Vision Systems Inc.（Vision Systems公司）；Sciopic Inc.（Sciopic公司）；Massachuessets Institute of Technology（麻省理工学院）
Monocular visual SLAM has become an attractive practical approach for robot localization and 3D environment mapping, since cameras are small, lightweight, inexpensive, and produce high-rate, high-resolution data streams. Although numerous robust tools have been developed, most existing systems are designed to operate in terrestrial environments and at relatively small scale (a few thousand frames) due to constraints on computation and storage. In this paper, we present a feature-based visual SLAM system for aerial video whose simple design permits near real-time operation, and whose scalabilit…
We consider the problem of automatically detecting small-scale solar photovoltaic arrays for behind-the-meter energy resource assessment in high resolution aerial imagery. Such algorithms offer a faster and more cost-effective solution to collecting information on distributed solar photovoltaic (PV) arrays, such as their location, capacity, and generated energy. The surface area of PV arrays, a characteristic which can be estimated from aerial imagery, provides an important proxy for array capacity and energy generation. In this work, we employ a state-of-the-art convolutional neural network a…
Unmanned Aerial Vehicle (UAV)-Based Mapping of Iris Pseudacorus L. Invasion in Laguna del Sauce (Uruguay) Coast
Universidad Católica del Uruguay（乌拉圭天主教大学）；Instituto de Investigacion en Ciencias de la Computacion (ICC), UBA-CONICET（计算机科学研究所（ICC），布宜诺斯艾利斯大学-阿根廷国家科学技术研究委员会）；Universidad de Buenos Aires（布宜诺斯艾利斯大学）
Unmanned Aerial vehicles (UAV) are a promising technology for smart farming related applications. Aerial monitoring of agriculture farms with UAV enables key decision-making pertaining to crop monitoring. Advancements in deep learning techniques have further enhanced the precision and reliability of aerial imagery based analysis. The capabilities to mount various kinds of sensors (RGB, spectral cameras) on UAV allows remote crop analysis applications such as vegetation classification and segmentation, crop counting, yield monitoring and prediction, crop mapping, weed detection, disease and nut…
This paper addresses outdoor terrain mapping using overhead images obtained from an unmanned aerial vehicle. Dense depth estimation from aerial images during flight is challenging. While feature-based localization and mapping techniques can deliver real-time odometry and sparse points reconstruction, a dense environment model is generally recovered offline with significant computation and storage. This paper develops a joint 2D-3D learning approach to reconstruct local meshes at each camera keyframe, which can be assembled into a global environment model. Each local mesh is initialized from sp…
The spread of the Red Palm Weevil has dramatically affected date growers, homeowners and governments, forcing them to deal with a constant threat to their palm trees. Early detection of palm tree infestation has been proven to be critical in order to allow treatment that may save trees from irreversible damage, and is most commonly performed by local physical access for individual tree monitoring. Here, we present a novel method for surveillance of Red Palm Weevil infested palm trees utilizing state-of-the-art deep learning algorithms, with aerial and street-level imagery data. To detect infes…
Department of Computer Science, the University of Alabama（阿拉巴马大学计算机科学系）
Flood extent mapping plays a crucial role in disaster management and national water forecasting. In recent years, high-resolution optical imagery becomes increasingly available with the deployment of numerous small satellites and drones. However, analyzing such imagery data to extract flood extent poses unique challenges due to the rich noise and shadows, obstacles (e.g., tree canopies, clouds), and spectral confusion between pixel classes (flood, dry) due to spatial heterogeneity. Existing machine learning techniques often focus on spectral and spatial features from raster images without full…
Sapienza University of Rome（罗马第一大学）；ETH Zurich（苏黎世联邦理工学院）
The combination of aerial survey capabilities of Unmanned Aerial Vehicles with targeted intervention abilities of agricultural Unmanned Ground Vehicles can significantly improve the effectiveness of robotic systems applied to precision agriculture. In this context, building and updating a common map of the field is an essential but challenging task. The maps built using robots of different types show differences in size, resolution and scale, the associated geolocation data may be inaccurate and biased, while the repetitiveness of both visual appearance and geometric structures found within ag…
Efficient On-Board Processing of Oblique UAV Video for Rapid Flood Extent Mapping
IDLab, Department of Information Technology, Ghent University - imec（IDLab，信息科技系，根特大学 - imec）；Environmental Intelligence Unit, VITO (Flemish Institute for Technological Research)（环境智能单元，VITO（佛兰德斯技术研究院））
Evaluation of Deep Learning Semantic Segmentation for Land Cover Mapping on Multispectral, Hyperspectral and High Spatial Aerial Imagery
Center for Space and Remote Sensing Research, National Central University（国立中央大学太空及遥测研究中心）；Department of Computer Science and Information Engineering, National Central University（国立中央大学资讯工程学系）
Aerial image analysis at a semantic level is important in many applications with strong potential impact in industry and consumer use, such as automated mapping, urban planning, real estate and environment monitoring, or disaster relief. The problem is enjoying a great interest in computer vision and remote sensing, due to increased computer power and improvement in automated image understanding algorithms. In this paper we address the task of automatic geolocalization of aerial images from recognition and matching of roads and intersections. Our proposed method is a novel contribution in the …
Rochester Institute of Technology（罗切斯特理工学院）；Chester F. Carlson Center for Imaging Science（切斯特·F·卡尔森成像科学中心）；School of Mathematical Sciences, Rochester Institute of Technology（罗切斯特理工学院数学科学学院）
Hyperspectral imaging holds enormous potential to improve the state-of-the-art in aerial vehicle tracking with low spatial and temporal resolutions. Recently, adaptive multi-modal hyperspectral sensors have attracted growing interest due to their ability to record extended data quickly from aerial platforms. In this study, we apply popular concepts from traditional object tracking, namely (1) Kernelized Correlation Filters (KCF) and (2) Deep Convolutional Neural Network (CNN) features to aerial tracking in hyperspectral domain. We propose the Deep Hyperspectral Kernelized Correlation Filter ba…
Deep learning-based object detection is a powerful approach for detecting faulty insulators in power lines. This involves training an object detection model from scratch, or fine tuning a model that is pre-trained on benchmark computer vision datasets. This approach works well with a large number of insulator images, but can result in unreliable models in the low data regime. The current literature mainly focuses on detecting the presence or absence of insulator caps, which is a relatively easy detection task, and does not consider detection of finer faults such as flashed and broken disks. In…
Reducing the gap between general purpose data and aerial images in concentrated solar power plants
organization= Department of Applied Mathematics II, University of Seville , city= Seville , country= Spain；organization= Multi-robot \& Control Systems group, University of Seville , city= Seville , country= Spain
Within commercial wind energy generation, the monitoring and predictive maintenance of wind turbine blades in-situ is a crucial task, for which remote monitoring via aerial survey from an Unmanned Aerial Vehicle (UAV) is commonplace. Turbine blades are susceptible to both operational and weather-based damage over time, reducing the energy efficiency output of turbines. In this study, we address automating the otherwise time-consuming task of both blade detection and extraction, together with fault detection within UAV-captured turbine blade inspection imagery. We propose BladeNet, an applicati…
A novel YOLO26-MoE optimized by an LLM agent for insulator fault detection considering UAV images
Department of Automation and Systems Engineering, Federal University of Santa Catarina, Florianópolis, Brazil（自动化与系统工程系，圣卡塔琳娜联邦大学，巴西弗洛里安波利斯）；Applications Lab, Faculty of Science, University of Salamanca, Plaza de los Caídos s/n, 37008 Salamanca, Spain（应用实验室，科学学院，萨拉曼卡大学，西班牙萨拉曼卡）
Real-Time Glass Detection and Reprojection using Sensor Fusion Onboard Aerial Robots
ARL DCIST CRA；NSF Grant CCR-2112665（NSF CCR-2112665）；NSF Graduate Research Fellowship Grant No. DGE-1845298（NSF DGE-1845298）；IoT4Ag Engineering Research Center
Institute of Systems Science, National University of Singapore（新加坡国立大学系统科学研究所）
In this paper, we propose an novel implementation of a simultaneous localization and mapping (SLAM) system based on a monocular camera from an unmanned aerial vehicle (UAV) using Depth prediction performed with Capsule Networks (CapsNet), which possess improvements over the drawbacks of the more widely-used Convolutional Neural Networks (CNN). An Extended Kalman Filter will assist in estimating the position of the UAV so that we are able to update the belief for the environment. Results will be evaluated on a benchmark dataset to portray the accuracy of our intended approach.
SLAM-Based Navigation and Fault Resilience in a Surveillance Quadcopter with Embedded Vision Systems
Guru Gobind Singh Indraprastha University（古鲁·戈宾德·辛格·因德拉普拉斯塔大学）；Delhi Skill and Entrepreneurship University (DSEU)（德里技能与创业大学（DSEU））；Delhi Institute of Tool Engineering（德里工具工程学院）
