Guided Attention Network for Object Detection and Counting on Drones
Institute of Software Chinese Academy of Sciences（中国科学院软件研究所）；University of Chinese Academy of Sciences（中国科学院大学）；University at Albany, SUNY（纽约州立大学奥尔巴尼分校）；JD Digits（京东数科）
Object detection and counting are related but challenging problems, especially for drone based scenes with small objects and cluttered background. In this paper, we propose a new Guided Attention Network (GANet) to deal with both object detection and counting tasks based on the feature pyramid. Different from the previous methods relying on unsupervised attention modules, we fuse different scales of feature maps by using the proposed weakly-supervised Background Attention (BA) between the background and objects for more semantic feature representation. Then, the Foreground Attention (FA) module is developed to consider both global and local appearance of the object to facilitate accurate localization. Moreover, the new data argumentation strategy is designed to train a robust model in various complex scenes. Extensive experiments on three challenging benchmarks (i.e., UAVDT, CARPK and PUCPR+) show the state-of-the-art detection and counting performance of the proposed method compared with existing methods.
National Taiwan University（国立台湾大学）；GE Global Research（通用电气全球研究院）
Existing counting methods often adopt regression-based approaches and cannot precisely localize the target objects, which hinders the further analysis (e.g., high-level understanding and fine-grained classification). In addition, most of prior work mainly focus on counting objects in static environments with fixed cameras. Motivated by the advent of unmanned flying vehicles (i.e., drones), we are interested in detecting and counting objects in such dynamic environments. We propose Layout Proposal Networks (LPNs) and spatial kernels to simultaneously count and localize target objects (e.g., car…
This paper proposes a novel approach to object detection on drone imagery, namely Multi-Proxy Detection Network with Unified Foreground Packing (UFPMP-Det). To deal with the numerous instances of very small scales, different from the common solution that divides the high-resolution input image into quite a number of chips with low foreground ratios to perform detection on them each, the Unified Foreground Packing (UFP) module is designed, where the sub-regions given by a coarse detector are initially merged through clustering to suppress background and the resulting ones are subsequently packe…
JD Finance America Corporation（京东金融美国公司）；Tianjin University（天津大学）
To promote the developments of object detection, tracking and counting algorithms in drone-captured videos, we construct a benchmark with a new drone-captured largescale dataset, named as DroneCrowd, formed by 112 video clips with 33,600 HD frames in various scenarios. Notably, we annotate 20,800 people trajectories with 4.8 million heads and several video-level attributes. Meanwhile, we design the Space-Time Neighbor-Aware Network (STNNet) as a strong baseline to solve object detection, tracking and counting jointly in dense crowds. STNNet is formed by the feature extraction module, followed …
LAF-YOLOv10 with Partial Convolution Backbone, Attention-Guided Feature Pyramid, Auxiliary P2 Head, and Wise-IoU Loss for Small Object Detection in Drone Aerial Imagery
Department of Information Technology（信息科技系）；Department of Computer Science and Engineering（计算机科学与工程系）
Omni Analytics Group（Omni分析集团）；Iowa State University（爱荷华州立大学）
Modern trends in digital agriculture have seen a shift towards artificial intelligence for crop quality assessment and yield estimation. In this work, we document how a parameter tuned single-shot object detection algorithm can be used to identify and count sorghum head from aerial drone images. Our approach involves a novel exploratory analysis that identified key structural elements of the sorghum images and motivated the selection of parameter-tuned anchor boxes that contributed significantly to performance. These insights led to the development of a deep learning model that outperformed th…
Monitoring of colonial waterbird nesting islands is essential to tracking waterbird population trends, which are used for evaluating ecosystem health and informing conservation management decisions. Recently, unmanned aerial vehicles, or drones, have emerged as a viable technology to precisely monitor waterbird colonies. However, manually counting waterbirds from hundreds, or potentially thousands, of aerial images is both difficult and time-consuming. In this work, we present a deep learning pipeline that can be used to precisely detect, count, and monitor waterbirds using aerial imagery coll…
Fraunhofer IOSB（弗劳恩霍夫光电、系统技术与图像利用研究所）；Kitware, Inc.（Kitware公司）；JD Finance America Corporation（京东金融美国公司）；Tianjin University（天津大学）；Stony Brook University（石溪大学）；University of Central Florida（中佛罗里达大学）；Huazhong University of Science and Technology（华中科技大学）；Northwestern Polytechnical University（西北工业大学）；Unisoc, Inc.（紫光展锐公司）；Karlsruhe Institute of Technology（卡尔斯鲁厄理工学院）；DeepBlue Technology(Shanghai) Co., Ltd（深兰科技（上海）有限公司）；Qatar University（卡塔尔大学）；Beijing University of Posts and Telecommunications（北京邮电大学）
Crowd counting on the drone platform is an interesting topic in computer vision, which brings new challenges such as small object inference, background clutter and wide viewpoint. However, there are few algorithms focusing on crowd counting on the drone-captured data due to the lack of comprehensive datasets. To this end, we collect a large-scale dataset and organize the Vision Meets Drone Crowd Counting Challenge (VisDrone-CC2020) in conjunction with the 16th European Conference on Computer Vision (ECCV 2020) to promote the developments in the related fields. The collected dataset is formed b…
Drones are employed in a growing number of visual recognition applications. A recent development in cell tower inspection is drone-based asset surveillance, where the autonomous flight of a drone is guided by localizing objects of interest in successive aerial images. In this paper, we propose a method to train deep weakly-supervised object localization (WSOL) models based only on image-class labels to locate object with high confidence. To train our localizer, pseudo labels are efficiently harvested from a self-supervised vision transformers (SSTs). However, since SSTs decompose the scene int…
Cross-modal Offset-guided Dynamic Alignment and Fusion for Weakly Aligned UAV Object Detection
State Key Laboratory of Optical Field Manipulation Science and Technology, Chinese Academy of Sciences（光学场操控科学与技术国家重点实验室，中国科学院）；Institute of Optics and Electronics, Chinese Academy of Sciences（中国科学院光电研究所）
Where to Perch in a Tree: Vision-Guidance for Tree-Grasping Drones
Department of Civil, Aerospace and Design Engineering, University of Bristol（布里斯托大学土木、航空航天与设计工程系）；Laboratory of Sustainability Robotics at Swiss Federal Laboratories for Materials Science and Technology (EMPA)（瑞士材料科学与技术联邦实验室可持续机器人实验室）；Ecole Polytechnique Fédérale de Lausanne (EPFL)（洛桑联邦理工学院）
Video Individual Counting and Tracking from Moving Drones: A Benchmark and Methods
School of Computer Science and Engineering, Sun Yat-sen University（中山大学计算机科学与工程学院）；school of Computer Science and Engineering, Hong Kong University of Science and Technology（香港科学与技术大学计算机科学与工程学院）；Department of Computer Science, City University of Hong Kong（香港城市大学计算机科学系）；School of Computer Science and Technology, Harbin Institute of Technology (Shenzhen)（哈尔滨工业大学（深圳）计算机科学与技术学院）；Chinese University of Hong Kong（香港中文大学）
DMM: Disparity-guided Multispectral Mamba for Oriented Object Detection in Remote Sensing
University of Electronic Science and Technology of China（电子科技大学）；Beijing Institute of Space Mechanics and Electricity（北京空间机电研究所）；Donghai Laboratory（东海实验室）
EGM-Det: Entropy-Guided Multimodal Adaptive Fusion for UAV RGB-IR Object Detection
School of Cybersecurity, Northwestern Polytechnical University（西北工业大学网络空间安全学院）；School of Automation and Software Engineering, Shanxi University（山西大学自动化与软件学院）
Detecting and Counting people in a human crowd from a moving drone present challenging problems that arisefrom the constant changing in the image perspective andcamera angle. In this paper, we test two different state-of-the-art approaches, density map generation with VGG19 trainedwith the Bayes loss function and detect-then-count with FasterRCNN with ResNet50-FPN as backbone, in order to comparetheir precision for counting and detecting people in differentreal scenarios taken from a drone flight. We show empiricallythat both proposed methodologies perform especially well fordetecting and coun…
JD Finance America Corporation（京东金融美国公司）；Tianjin University（天津大学）；University at Albany, State University of New York（纽约州立大学奥尔巴尼分校）
This paper proposes a space-time multi-scale attention network (STANet) to solve density map estimation, localization and tracking in dense crowds of video clips captured by drones with arbitrary crowd density, perspective, and flight altitude. Our STANet method aggregates multi-scale feature maps in sequential frames to exploit the temporal coherency, and then predict the density maps, localize the targets, and associate them in crowds simultaneously. A coarse-to-fine process is designed to gradually apply the attention module on the aggregated multi-scale feature maps to enforce the network …
Current crowd counting algorithms are only concerned about the number of people in an image, which lacks low-level fine-grained information of the crowd. For many practical applications, the total number of people in an image is not as useful as the number of people in each sub-category. E.g., knowing the number of people waiting inline or browsing can help retail stores; knowing the number of people standing/sitting can help restaurants/cafeterias; knowing the number of violent/non-violent people can help police in crowd management. In this paper, we propose fine-grained crowd counting, which…
Rotating object detection has wide applications in aerial photographs, remote sensing images, UAVs, etc. At present, most of the rotating object detection datasets focus on the field of remote sensing, and these images are usually shot in high-altitude scenes. However, image datasets captured at low-altitude areas also should be concerned, such as drone-based datasets. So we present a low-altitude dronebased dataset, named UAV-ROD, aiming to promote the research and development in rotating object detection and UAV applications. The UAV-ROD consists of 1577 images and 30,090 instances of car ca…
SLGNet: Synergizing Structural Priors and Language-Guided Modulation for Multimodal Object Detection
Aerospace Information Research Institute, Chinese Academy of Sciences（中国科学院航天信息研究所）；Key Laboratory of Target Cognition and Application Technology, Chinese Academy of Sciences（中国科学院目标认知与应用技术重点实验室）；School of Electronic, Electrical and Communication Engineering, University of Chinese Academy of Sciences（中国科学院大学电子电气与通信工程学院）；School of Software Engineering, Xi’an Jiaotong University（西安交通大学软件学院）
UniGeo: A Multi-modal Large Language Model for Text-Guided Cross-View Geo-Localization
School of Computer Engineering and Science, Shanghai University（上海大学计算机工程与科学学院）；Institute of Collaborative Innovation, University of Macau（澳门大学协同创新研究院）
We present BEAMER: a new spatially exploitative approach to learning object detectors which shows excellent results when applied to the task of detecting objects in greyscale aerial imagery in the presence of ambiguous and noisy data. There are four main contributions used to produce these results. First, we introduce a grammar-guided feature extraction system, enabling the exploration of a richer feature space while constraining the features to a useful subset. This is specified with a rule-based generative grammar crafted by a human expert. Second, we learn a classifier on this data using a …
Crowd analysis from drones has attracted increasing attention in recent times due to the ease of use and affordable cost of these devices. However, how this technology can provide a solution to crowd flow detection is still an unexplored research question. To this end, we propose a crowd flow detection method for video sequences shot by a drone. The method is based on a fully-convolutional network that learns to perform crowd clustering in order to detect the centroids of crowd-dense areas and track their movement in consecutive frames. The proposed method proved effective and efficient when t…
In this paper, we address the challenging problem of crowd counting in congested scenes. Specifically, we present Inverse Attention Guided Deep Crowd Counting Network (IA-DCCN) that efficiently infuses segmentation information through an inverse attention mechanism into the counting network, resulting in significant improvements. The proposed method, which is based on VGG-16, is a single-step training framework and is simple to implement. The use of segmentation information results in minimal computational overhead and does not require any additional annotations. We demonstrate the significanc…
Sun Yat-sen University（中山大学）；Harbin Institute of Technology (Shenzhen)（哈尔滨工业大学（深圳））；Hong Kong University of Science and Technology（香港科学大学）；City University of Hong Kong（香港城市大学）
DEPFusion: Dual-Domain Enhancement and Priority-Guided Mamba Fusion for UAV Multispectral Object Detection
School of Information Engineering, Guangdong University of Technology（广东工业大学信息工程学院）；School of Electronic and Information Engineering, South China University of Technology（华南理工大学电子信息学院）；School of Artificial Intelligence, XiDian University（西安电子科技大学人工智能学院）
Towards Natural Language-Guided Drones: GeoText-1652 Benchmark with Spatial Relation Matching
School of Computing, National University of Singapore（新加坡国立大学计算机学院）；FST and ICI, University of Macau（澳门大学科技学院与当代计算机与软件研究所）；School of Communication Engineering, Hangzhou Dianzi University（杭州电子科技大学通信工程学院）
Beyond Image-Text Matching: Verb Understanding in Multimodal Transformers Using Guided Masking
Faculty of Information Technology, Brno University of Technology（布尔诺理工大学信息技术学院）；Kempelen Institute of Intelligent Technologies（肯佩伦智能技术研究所）；George Mason University（乔治梅森大学）
École Polytechnique Fédérale de Lausanne (EPFL)（洛桑联邦理工学院）
State-of-the-art methods for counting people in crowded scenes rely on deep networks to estimate crowd density in the image plane. While useful for this purpose, this image-plane density has no immediate physical meaning because it is subject to perspective distortion. This is a concern in sequences acquired by drones because the viewpoint changes often. This distortion is usually handled implicitly by either learning scale-invariant features or estimating density in patches of different sizes, neither of which accounts for the fact that scale changes must be consistent over the whole scene. I…
Object counting, which aims to count the accurate number of object instances in images, has been attracting more and more attention. However, challenges such as large scale variation, complex background interference, and non-uniform density distribution greatly limit the counting accuracy, particularly striking in remote sensing imagery. To mitigate the above issues, this paper proposes a novel framework for dense object counting in remote sensing images, which incorporates a pyramidal scale module (PSM) and a global context module (GCM), dubbed PSGCNet, where PSM is used to adaptively capture…
Crowd counting aims to estimate the number of persons in a scene. Most state-of-the-art crowd counting methods based on color images can't work well in poor illumination conditions due to invisible objects. With the widespread use of infrared cameras, crowd counting based on color and thermal images is studied. Existing methods only achieve multi-modal fusion without count objective constraint. To better excavate multi-modal information, we use count-guided multi-modal fusion and modal-guided count enhancement to achieve the impressive performance. The proposed count-guided multi-modal fusion …
Harbin Institute of Technology（哈尔滨工业大学）；Baidu Inc.（百度公司）；Peng Cheng Laboratory（鹏城实验室）
In this paper, we propose a novel perspective-guided convolution (PGC) for convolutional neural network (CNN) based crowd counting (i.e. PGCNet), which aims to overcome the dramatic intra-scene scale variations of people due to the perspective effect. While most state-of-the-arts adopt multi-scale or multi-column architectures to address such issue, they generally fail in modeling continuous scale variations since only discrete representative scales are considered. PGCNet, on the other hand, utilizes perspective information to guide the spatially variant smoothing of feature maps before feedin…
In real-world crowd counting applications, the crowd densities in an image vary greatly. When facing density variation, humans tend to locate and count the targets in low-density regions, and reason the number in high-density regions. We observe that CNN focus on the local information correlation using a fixed-size convolution kernel and the Transformer could effectively extract the semantic crowd information by using the global self-attention mechanism. Thus, CNN could locate and estimate crowds accurately in low-density regions, while it is hard to properly perceive the densities in high-den…
The Hong Kong University of Science and Technology（香港科技大学）
We study video crowd counting, which is to estimate the number of objects (people in this paper) in all the frames of a video sequence. Previous work on crowd counting is mostly on still images. There has been little work on how to properly extract and take advantage of the spatial-temporal correlation between neighboring frames in both short and long ranges to achieve high estimation accuracy for a video sequence. In this work, we propose Monet, a novel and highly accurate motion-guided non-local spatial-temporal network for video crowd counting. Monet first takes people flow (motion informat…
Carnegie Mellon University（卡内基梅隆大学）；Chongqing University of Posts and Telecommunications（重庆邮电大学）；Xi’an Jiaotong University（西安交通大学）
In real-world crowd counting applications, the crowd densities vary greatly in spatial and temporal domains. A detection based counting method will estimate crowds accurately in low density scenes, while its reliability in congested areas is downgraded. A regression based approach, on the other hand, captures the general density information in crowded regions. Without knowing the location of each person, it tends to overestimate the count in low density areas. Thus, exclusively using either one of them is not sufficient to handle all kinds of scenes with varying densities. To address this issu…
Does it Really Count? Assessing Semantic Grounding in Text-Guided Class-Agnostic Counting
Institute of Information Science and Technologies of the National Research Council (ISTI-CNR)（意大利国家研究理事会信息科学与技术研究所）；University of Pisa - Department of Information Engineering（比萨大学信息工程系）
Prince Sultan University（苏丹王子大学）；CISTER, INESC-TEC, ISEP, Polytechnic Institute of Porto（波尔图理工学院ISEP的INESC-TEC旗下CISTER（注：因原文为层级关联机构，按要求提取到学院/研究院级别，整合为：波尔图理工学院）
In this paper, we propose a deep learning framework for the automated counting and geolocation of palm trees from aerial images using convolutional neural networks. For this purpose, we collected aerial images in a palm tree Farm in the Kharj region, in Riyadh Saudi Arabia, using DJI drones, and we built a dataset of around 10,000 instances of palms trees. Then, we developed a convolutional neural network model using the state-of-the-art, Faster R-CNN algorithm. Furthermore, using the geotagged metadata of aerial images, we used photogrammetry concepts and distance corrections to detect the ge…
