Understanding the complex urban infrastructure with centimeter-level accuracy is essential for many applications from autonomous driving to mapping, infrastructure monitoring, and urban management. Aerial images provide valuable information over a large area instantaneously; nevertheless, no current dataset captures the complexity of aerial scenes at the level of granularity required by real-world applications. To address this, we introduce SkyScapes, an aerial image dataset with highly-accurate, fine-grained annotations for pixel-level semantic labeling. SkyScapes provides annotations for 31 semantic categories ranging from large structures, such as buildings, roads and vegetation, to fine details, such as 12 (sub-)categories of lane markings. We have defined two main tasks on this dataset: dense semantic segmentation and multi-class lane-marking prediction. We carry out extensive experiments to evaluate state-of-the-art segmentation methods on SkyScapes. Existing methods struggle to deal with the wide range of classes, object sizes, scales, and fine details present. We therefore propose a novel multi-task model, which incorporates semantic edge detection and is better tuned for feature extraction from a wide range of scales. This model achieves notable improvements over the baselines in region outlines and level of detail on both tasks.
THYME: Temporal Hierarchical-Cyclic Interactivity Modeling for Video Scene Graphs in Aerial Footage
University of Arkansas（亚拉巴马大学）；University of Science, VNU-HCM（科学大学，VNU-HCM）；Vietnam National University, Ho Chi Minh（越南国家大学，胡志明市）；Ohio State University（俄亥俄州立大学）
Classification between thousands of classes in high-resolution images is one of the heavily studied problems in deep learning over the last decade. However, the challenge of fine-grained multi-class classification of objects in aerial images, especially in low resource cases, is still challenging and an active area of research in the literature. Solving this problem can give rise to various applications in the field of scene understanding and classification and re-identification of specific objects from aerial images. In this paper, we provide a description of our dataset - COFGA of multi-clas…
School of Computer Science, Wuhan University（武汉大学计算机学院）；School of Artificial Intelligence, Wuhan University（武汉大学人工智能学院）；School of Electronic Information, Wuhan University（武汉大学电子信息学院）
Israel Ministry Of Defense’s Directorate of Defense Research and Development (IMOD’s DDR&D)（以色列国防部国防研究与开发局 (IMOD DDR&D)）
Detection and classification of objects in overhead images are two important and challenging problems in computer vision. Among various research areas in this domain, the task of fine-grained classification of objects in overhead images has become ubiquitous in diverse real-world applications, due to recent advances in high-resolution satellite and airborne imaging systems. The small inter-class variations and the large intra class variations caused by the fine grained nature make it a challenging task, especially in low-resource cases. In this paper, we introduce COFGA a new open dataset for …
AeroRAG: Structured Multimodal Retrieval-Augmented LLM for Fine-Grained Aerial Visual Reasoning
Research Center for Space Computing System, Zhejiang Lab, Hangzhou（杭州浙大实验室空间计算系统研究中心）；Hangzhou Institute for Advanced Study, University of Chinese Academy of Sciences, Hangzhou（中国科学院大学杭州高等研究院）；School of Cyber Science and Engineering, Zhengzhou University（郑州大学计算机科学与工程学院）；School of Computer Science and Technology, Xi’an Jiaotong University（西安交通大学计算机科学与技术学院）
Qatar Computing Research Institute（卡塔尔计算研究所）；Hamad bin Khalifa University（哈马德·本·哈利法大学）
We propose Nazr-CNN1, a deep learning pipeline for object detection and fine-grained classification in images acquired from Unmanned Aerial Vehicles (UAVs) for damage assessment and monitoring. Nazr-CNN consists of two components. The function of the first component is to localize objects (e.g. houses or infrastructure) in an image by carrying out a pixel-level classification. In the second component, a hidden layer of a Convolutional Neural Network (CNN) is used to encode Fisher Vectors (FV) of the segments generated from the first component in order to help discriminate between different lev…
University of Science and Technology of China（中国科学技术大学）；Ping An Technology Co., Ltd（平安科技有限公司）；Hefei ZhanDa Intelligence Technology Co., Ltd（合肥展大智能科技有限公司）
Multi-modal aerial view object classification (MAVOC) in Automatic target recognition (ATR), although an important and challenging problem, has been under studied. This paper firstly finds that fine-grained data, class imbalance and various shooting conditions preclude the representational ability of general image classification. Moreover, the MAVOC dataset has scene aggregation characteristics. By exploiting these properties, we propose Scene Clustering Based Pseudo-labeling Strategy (SCP-Label), a simple yet effective method to employ in post-processing. The SCP-Label brings greater accuracy…
OpenCVL: An Open, Diverse, and Large-Scale Dataset for Fine-Grained Cross-View Localization
École Polytechnique Fédérale de Lausanne (EPFL)（洛桑联邦理工学院）；Southern University of Science and Technology (SUSTech)（南方科技大学）；Delft University of Technology（代尔夫特理工大学）；Zenseact（Zenseact公司）
Beijing University of Chemical Technology（北京化工大学）；College of Information Science & Technology（信息科学与技术学院）
For the task of subdecimeter aerial imagery segmentation, fine-grained semantic segmentation results are usually difficult to obtain because of complex remote sensing content and optical conditions. Recently, convolutional neural networks (CNNs) have shown outstanding performance on this task. Although many deep neural network structures and techniques have been applied to improve the accuracy, few have paid attention to better differentiating the easily confused classes. In this paper, we propose TreeSegNet which adopts an adaptive network to increase the classification rate at the pixelwise …
Adapting Fine-Grained Cross-View Localization to Areas without Fine Ground Truth
École Polytechnique Fédérale de Lausanne (EPFL)（洛桑联邦理工学院）；ShanghaiTech University（上海科技大学）；Australian National University（澳大利亚国立大学）；Delft University of Technology（代尔夫特理工大学）
RefAerial: A Benchmark and Approach for Referring Detection in Aerial Images
School of Artificial Intelligence, Anhui University（安徽大学人工智能学院）；School of Computer Science and Technology, Anhui University（安徽大学计算机科学与技术学院）；State Key Laboratory of Opto-Electronic Information Acquisition and Protection Technology, Anhui University（安徽大学光电信息采集与防护技术国家重点实验室）；Anhui Provincial Key Laboratory of Security Artificial Intelligence, Anhui University（安徽省安全人工智能重点实验室）；Anhui Provincial Key Laboratory of Multimodal Cognitive Computation, Anhui University（安徽省多模态认知计算重点实验室）
Cross-modal Fuzzy Alignment Network for Text-Aerial Person Retrieval and A Large-scale Benchmark
State Key Laboratory of Opto-Electronic Information Acquisition and Protection Technology（光电信息采集与防护技术国家重点实验室）；School of Computer Science and Technology, Anhui University（安徽大学计算机科学与技术学院）；School of Artificial Intelligence, Anhui University（安徽大学人工智能学院）；The University of Hong Kong（香港大学）
Unifying UAV Cross-View Geo-Localization via 3D Geometric Perception
School of Electronic Information, Wuhan University（武汉大学电子信息学院）；School of Artificial Intelligence, Wuhan University（武汉大学人工智能学院）；Technology and Engineering Center for Space Utilization and the Key Laboratory of Space Utilization, Chinese Academy of Sciences（中国科学院空间应用工程与技术中心暨空间利用重点实验室）；School of Aeronautics and Astronautics, University of Chinese Academy of Sciences（中国科学院大学航空宇航学院）
Graph-based Semantic Calibration Network for Unaligned UAV RGBT Image Semantic Segmentation and A Large-scale Benchmark
Key Laboratory of Intelligent Computing & Signal Processing (Anhui University), Ministry of Education（智能计算与信号处理重点实验室（安徽大学），教育部）；Anhui Provincial Key Laboratory of Multimodal Cognitive Computation, School of Artificial Intelligence, Anhui University（安徽省多模态认知计算重点实验室，人工智能学院，安徽大学）；School of Computer Science and Technology, Anhui University（计算机科学与技术学院，安徽大学）；GEOVIS Earth Technology Co., Ltd.（GEOVIS地球科技有限公司）
STPLS3D: A Large-Scale Synthetic and Real Aerial Photogrammetry 3D Point Cloud Dataset
University of Southern California（南加州大学）；University of Oxford（牛津大学）；Arizona State University（亚利桑那州立大学）；University of Toronto（多伦多大学）；Carnegie Mellon University（卡内基梅隆大学）
X$^2$Localizer: Cross-grained Alignment for Progressive Cross-view Video Geo-localization
University College London（伦敦大学学院）；Karlsruhe Institute of Technology（卡尔斯鲁厄理工学院）；Hunan University（湖南大学）；University of Alberta（阿尔伯塔大学）；Shenzhen University（深圳大学）
FTDMamba: Frequency-Assisted Temporal Dilation Mamba for Unmanned Aerial Vehicle Video Anomaly Detection
Information Materials and Intelligent Sensing Laboratory of Anhui Province（安徽省信息材料与智能感知实验室）；Anhui Provincial Key Laboratory of Multimodal Cognitive Computation（安徽省多模态认知计算重点实验室）；School of Computer Science and Technology, Anhui University（安徽大学计算机科学与技术学院）；Department of Computer Science and Engineering, The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳）计算机科学与工程系）
Fine-grained object recognition that aims to identify the type of an object among a large number of subcategories is an emerging application with the increasing resolution that exposes new details in image data. Traditional fully supervised algorithms fail to handle this problem where there is low between-class variance and high within-class variance for the classes of interest with small sample sizes. We study an even more extreme scenario named zero-shot learning (ZSL) in which no training example exists for some of the classes. ZSL aims to build a recognition model for new unseen categories…
GRASP: Granularity-Aware Region Alignment and Semantic Prototype Learning for Fine-Grained Cross-Modal Understanding in Drone Views
Aerospace Information Research Institute, Chinese Academy of Sciences（中国科学院空天信息创新研究院）；University of Chinese Academy of Sciences（中国科学院大学）；School of Electronic, Electrical and Communication Engineering, University of Chinese Academy of Sciences（中国科学院大学电子电气与通信工程学院）
OS-W2S: An Automatic Labeling Engine for Language-Guided Open-Set Aerial Object Detection
Nanjing University of Science and Technology（南京理工大学）；Intellifusion Inc.（Intellifusion公司）；Northwestern Polytechnical University（西北工业大学）；Zhejiang Lab（浙江实验室）；Yan’an University（延安大学）；Beijing University of Posts and Telecommunications（北京邮电大学）
Memory-Augmented Multimodal Large Language Models for Small Object Understanding in Streaming Aerial Videos
The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；University of Freiburg（弗莱堡大学）；Hangzhou City University（杭州城市大学）；XGRIDS（XGRIDS公司）；Fudan University（复旦大学）；Hong Kong Baptist University（香港浸会大学）
Revisiting Evaluation Metrics for Semantic Segmentation: Optimization and Evaluation of Fine-grained Intersection over Union
KU Leuven（鲁汶大学）；Google（谷歌公司）；Google DeepMind（谷歌DeepMind）；University of Oxford（牛津大学）；EPFL（洛桑联邦理工学院）；ETH Zürich（苏黎世联邦理工学院）；INSAIT（INSAIT研究院）；Samsung Research（三星研究院）
AGSPNet: A framework for parcel-scale crop fine-grained semantic change detection from UAV high-resolution imagery with agricultural geographic scene constraints
Hunan University of Science and Technology（湖南科技大学）
FineSkiing: A Fine-grained Benchmark for Skiing Action Quality Assessment
Key Laboratory of Symbolic Computation and Knowledge Engineering of Ministry of Education, College of Computer Science and Technology（教育部符号计算与知识工程重点实验室、计算机科学与技术学院）；Jilin University（吉林大学）；BNRist, THUIBCS, BLBCI, School of Software（BNRist、THUIBCS、BLBCI、软件学院）；Tsinghua University（清华大学）
California Institute of Technology（加州理工学院）；ETH Zürich（苏黎世联邦理工学院）
Up-to-date catalogs of the urban tree population are important for municipalities to monitor and improve quality of life in cities. Despite much research on automation of tree mapping, mainly relying on dedicated airborne LiDAR or hyperspectral campaigns, trees are still mostly mapped manually in practice. We present a fully automated tree detection and species recognition pipeline to process thousands of trees within a few hours using publicly available aerial and street view images of Google MapsTM. These data provide rich information (viewpoints, scales) from global tree shapes to bark text…
UAV-Flow Colosseo: A Real-World Benchmark for Flying-on-a-Word UAV Imitation Learning
Institute of Artificial Intelligence, Beihang University（北京航空航天大学人工智能学院）；National University of Singapore（新加坡国立大学）；MMLab, CUHK（香港中文大学MMLab）；Hangzhou International Innovation Institute of Beihang University（北京航空航天大学杭州国际创新研究院）
