Semantic segmentation for aerial platforms has been one of the fundamental scene understanding task for the earth observation. Most of the semantic segmentation research focused on scenes captured in nadir view, in which objects have relatively smaller scale variation compared with scenes captured in oblique view. The huge scale variation of objects in oblique images limits the performance of deep neural networks (DNN) that process images in a single scale fashion. In order to tackle the scale variation issue, in this paper, we propose the novel bidirectional multi-scale attention networks, which fuse features from multiple scales bidirectionally for more adaptive and effective feature extraction. The experiments are conducted on the UAVid2020 dataset and have shown the effectiveness of our method. Our model achieved the state-of-the-art (SOTA) result with a mean intersection over union (mIoU) score of 70.80%.
University of Twente（特文特大学）；Wuhan University（武汉大学）；Ohio State University（俄亥俄州立大学）
Semantic segmentation has been one of the leading research interests in computer vision recently. It serves as a perception foundation for many fields, such as robotics and autonomous driving. The fast development of semantic segmentation attributes enormously to the large scale datasets, especially for the deep learning related methods. There already exist several semantic segmentation datasets for comparison among semantic segmentation methods in complex urban scenes, such as the Cityscapes and CamVid datasets, where the side views of the objects are captured with a camera mounted on the dri…
Fraunhofer IOSB（弗劳恩霍夫IOSB）；Karlsruhe Institute of Technology（卡尔斯鲁厄理工学院）
With the technological advancements of aerial imagery and accurate 3d reconstruction of urban environments, more and more attention has been paid to the automated analyses of urban areas. In our work, we examine two important aspects that allow live analysis of building structures in city models given oblique aerial imagery, namely automatic building extraction with convolutional neural networks (CNNs) and selective real-time depth estimation from aerial imagery. We use transfer learning to train the Faster R-CNN method for real-time deep object detection, by combining a large ground-based dat…
ObliCity: A Benchmark and Baseline for Roof-to-Ground Projection Displacement Correction
School of Electronic, Electrical and Communication Engineering, University of Chinese Academy of Sciences（中国科学院大学电子电气与通信工程学院）；College of Computing, Department of Data Science, City University of Hong Kong（香港城市大学计算学院数据科学系）；Aerospace Information Research Institute, Chinese Academy of Sciences（中国科学院空天信息创新研究院）；Department of Electrical and Computer Engineering, The University of Hong Kong（香港大学电气与计算机工程系）
University of Stuttgart（斯图加特大学）；Leibniz University Hannover（汉诺威莱布尼茨大学）；University of Zurich（苏黎世大学）；ETH Zurich（苏黎世联邦理工学院）；Delft University of Technology（代尔夫特理工大学）
Automated semantic segmentation and object detection are of great importance in geospatial data analysis. However, supervised machine learning systems such as convolutional neural networks require large corpora of annotated training data. Especially in the geospatial domain, such datasets are quite scarce. Within this paper, we aim to alleviate this issue by introducing a new annotated 3D dataset that is unique in three ways: i) The dataset consists of both an Unmanned Aerial Vehicle (UAV) laser scanning point cloud and a 3D textured mesh. ii) The point cloud features a mean point density of a…
Depth Supervised Neural Surface Reconstruction from Airborne Imagery
nFrames ESRI；Institute for Photogrammetry and Geoinformatics, University of Stuttgart（测绘与地理信息研究所，斯图加特大学）；Geoinformatics, University of Stuttgart（地理信息学，斯图加特大学）
FaSS-MVS -- Fast Multi-View Stereo with Surface-Aware Semi-Global Matching from UAV-borne Monocular Imagery
Fraunhofer Institute of Optronics, System Technologies and Image Exploitation (IOSB)（弗劳恩霍夫光学技术、系统技术与图像利用研究所）；Fraunhofer Center for Machine Learning（弗劳恩霍夫机器学习中心）；Karlsruhe Institute of Technology (KIT)（卡尔斯鲁厄理工学院）
University of Montana（蒙大拿大学）；United States Geological Survey（美国地质调查局）
We develop a probabilistic method for tracking glacier surface motion based on time-lapse imagery, which works by sequentially resampling a stochastic state-space model according to a likelihood determined through correlation between reference and test images. The method is robust due to its natural handling of periodic occlusion and its capacity to follow multiple hypothesis displacements between images, and can improve estimates of velocity magnitude and direction through the inclusion of observations from an arbitrary number of cameras. We apply the method to an annual record of images from…
BRF-GS: Hyperspectral Bidirectional Reflectance Factor Modeling and Image Generation Based on 3D Gaussian Splatting
Aerospace Information Research Institute, Chinese Academy of Sciences（中国科学院空天信息创新研究院）；International Research Center of Big Data for Sustainable Development Goals（国际可持续发展大数据研究中心）；University of Chinese Academy of Sciences（中国科学院大学）
This work presents an Artificial Intelligence (AI) system, based on the Faster Region-Based Convolution Neural Network (Faster R-CNN) framework, which detects and counts apples from oblique, aerial drone imagery of giant commercial orchards. To reduce computational cost, a novel precursory stage to the network is designed to preprocess raw imagery into cropped images of individual trees. Unique geospatial identifiers are allocated to these using the perspective projection model. This employs Real-Time Kinematic (RTK) data, Digital Terrain and Surface Models (DTM and DSM), as well as internal a…
Northeast Electric Power University（东北电力大学）；Changchun University of Science and Technology（长春理工大学）；University of Sydney（悉尼大学）
Recognition accuracy and response time are both critically essential ahead of building practical electroencephalography (EEG) based brain-computer interface (BCI). Recent approaches, however, have either compromised in the classification accuracy or responding time. This paper presents a novel deep learning approach designed towards remarkably accurate and responsive motor imagery (MI) recognition based on scalp EEG. Bidirectional Long Short-term Memory (BiLSTM) with the Attention mechanism manages to derive relevant features from raw EEG signals. The connected graph convolutional neural netwo…
Fraunhofer IOSB（弗劳恩霍夫IOSB研究所）；Karlsruhe Institute of Technology（卡尔斯鲁厄理工学院）
Online augmentation of an oblique aerial image sequence with structural information is an essential aspect in the process of 3D scene interpretation and analysis. One key aspect in this is the efficient dense image matching and depth estimation. Here, the Semi-Global Matching (SGM) approach has proven to be one of the most widely used algorithms for efficient depth estimation, providing a good trade-off between accuracy and computational complexity. However, SGM only models a first-order smoothness assumption, thus favoring fronto-parallel surfaces. In this work, we present a hierarchical algo…
PixDLM: A Dual-Path Multimodal Language Model for UAV Reasoning Segmentation
Key Laboratory of Multimedia Trusted Perception and Efficient Computing, Ministry of Education of China, Xiamen University（中国教育部多媒体可信感知与高效计算重点实验室，厦门大学）
Unifying UAV Cross-View Geo-Localization via 3D Geometric Perception
School of Electronic Information, Wuhan University（武汉大学电子信息学院）；School of Artificial Intelligence, Wuhan University（武汉大学人工智能学院）；Technology and Engineering Center for Space Utilization and the Key Laboratory of Space Utilization, Chinese Academy of Sciences（中国科学院空间应用工程与技术中心暨空间利用重点实验室）；School of Aeronautics and Astronautics, University of Chinese Academy of Sciences（中国科学院大学航空宇航学院）
GeoBridge++: Fact-Guided Geo-Semantic Bridging for Unified Cross-View Geo-Localization
School of Computer Science and Technology, Jilin University（吉林大学计算机科学与技术学院）；School of Computer Science, Wuhan University（武汉大学计算机学院）；Zhongguancun Academy, Beijing, China（中关村学院，北京，中国）；State Key Laboratory of Information Engineering in Surveying, Mapping and Remote Sensing, Wuhan University（武汉大学测绘遥感信息工程国家重点实验室）
ReLATE: Reliability-Guided Evidence Fusion for Robust UAV--Satellite cross-view Geo-Localization
School of Electronics and Information Engineering, Harbin Institute of Technology（哈尔滨工业大学电子与信息工程学院）；National Key Laboratory of Radar Detection and Sensing, Nanjing Research Institute of Electronics Technology（南京电子技术研究所雷达探测与感知国家重点实验室）；School of Electrical and Electronic Engineering, University of Manchester（曼彻斯特大学电气与电子工程学院）
UAV traffic scene understanding: A regulation embedded multi-modal network and a unified benchmark
Computer Network Information Center, Chinese Academy of Sciences, Beijing 100190, China（中国科学院计算机网络信息中心，北京100190，中国）；University of Chinese Academy of Sciences, Beijing 100190, China（中国科学院大学，北京100190，中国）；Anhui Provincial Key Laboratory of Multi-modal Cognitive Computation, Anhui University, Hefei 230601, China（安徽省多模态认知计算重点实验室，安徽大学，合肥230601，中国）；Information Materials and Intelligent Sensing Laboratory of Anhui Province, Anhui University, Hefei 230601, China（安徽省信息材料与智能感知实验室，安徽大学，合肥230601，中国）
