FPN-based detectors have made significant progress in general object detection, e.g., MS COCO and PASCAL VOC. However, these detectors fail in certain application scenarios, e.g., tiny object detection. In this paper, we argue that the top-down connections between adjacent layers in FPN bring two-side influences for tiny object detection, not only positive. We propose a novel concept, fusion factor, to control information that deep layers deliver to shallow layers, for adapting FPN to tiny object detection. After series of experiments and analysis, we explore how to estimate an effective value of fusion factor for a particular dataset by a statistical method. The estimation is dependent on the number of objects distributed in each layer. Comprehensive experiments are conducted on tiny object detection datasets, e.g., TinyPerson and Tiny CityPersons. Our results show that when configuring FPN with a proper fusion factor, the network is able to achieve significant performance gains over the baseline on tiny object detection datasets. Codes and models will be released.
A DeNoising FPN With Transformer R-CNN for Tiny Object Detection
Department of Electronics and Electrical Engineering, National Yang Ming Chiao Tung University（电子工程系，阳明交通大学）；Institute of Electronics, National Yang Ming Chiao Tung University（电子研究所，阳明交通大学）；Department of Computer Science and Information Engineering, National Taiwan University (NTU)（计算机科学与信息工程系，国立台湾大学）
Learning pyramidal feature representations is crucial for recognizing object instances at different scales. Feature Pyramid Network (FPN) is the classic architecture to build a feature pyramid with high-level semantics throughout. However, intrinsic defects in feature extraction and fusion inhibit FPN from further aggregating more discriminative features. In this work, we propose Attention Aggregation based Feature Pyramid Network (A^2-FPN), to improve multi-scale feature learning through attention-guided feature aggregation. In feature extraction, it extracts discriminative features by collec…
National Central University（中央大学）；National Yang Ming Chiao Tung University（阳明交通大学）
FPN (Feature Pyramid Network) has become a basic component of most SoTA one stage object detectors. Many previous studies have repeatedly proved that FPN can caputre better multi-scale feature maps to more precisely describe objects if they are with different sizes. However, for most backbones such VGG, ResNet, or DenseNet, the feature maps at each layer are downsized to their quarters due to the pooling operation or convolutions with stride 2. The gap of down-scaling-by-2 is large and makes its FPN not fuse the features smoothly. This paper proposes a new SFPN (Synthetic Fusion Pyramid Networ…
Wuhan University（武汉大学）；Lancaster University（兰卡斯特大学）；UK Centre for Ecology and Hydrology（英国生态与水文中心）
Semantic segmentation using fine-resolution remotely sensed images plays a critical role in many practical applications, such as urban planning, environmental protection, natural and anthropogenic landscape monitoring, etc. However, the automation of semantic segmentation, i.e., automatic categorization/labeling and segmentation is still a challenging task, particularly for fine-resolution images with huge spatial and spectral complexity. Addressing such a problem represents an exciting research field, which paves the way for scene-level landscape pattern analysis and decision making. In this …
School of Computer Science and Technology, Huazhong University of Science and Technology（华中科技大学计算机科学与技术学院）；School of Computer and Information Engineering, Henan University（河南大学计算机与信息工程学院）
Feature pyramid network (FPN) has been an effective framework to extract multi-scale features in object detection. However, current FPN-based methods mostly suffer from the intrinsic flaw of channel reduction, which brings about the loss of semantical information. And the miscellaneous fused feature maps may cause serious aliasing effects. In this paper, we present a novel channel enhancement feature pyramid network (CE-FPN) with three simple yet effective modules to alleviate these problems. Specifically, inspired by sub-pixel convolution, we propose a sub-pixel skip fusion method to perform …
Tsinghua University（清华大学）；University of Michigan（密歇根大学）
3D object detection with a single image is an essential and challenging task for autonomous driving. Recently, keypoint-based monocular 3D object detection has made tremendous progress and achieved great speed-accuracy trade-off. However, there still exists a huge gap with LIDAR-based methods in terms of accuracy. To improve their performance without sacrificing efficiency, we propose a sort of lightweight feature pyramid network called Lite-FPN to achieve multi-scale feature fusion in an effective and efficient way, which can boost the multi-scale detection capability of keypoint-based detect…
Current state-of-the-art detectors typically exploit feature pyramid to detect objects at different scales. Among them, FPN is one of the representative works that build a feature pyramid by multi-scale features summation. However, the design defects behind prevent the multi-scale features from being fully exploited. In this paper, we begin by first analyzing the design defects of feature pyramid in FPN, and then introduce a new feature pyramid architecture named AugFPN to address these problems. Specifically, AugFPN consists of three components: Consistent Supervision, Residual Feature Augmen…
Object detection has made great progress in the past few years along with the development of deep learning. However, most current object detection methods are resource hungry, which hinders their wide deployment to many resource restricted usages such as usages on always-on devices, battery-powered low-end devices, etc. This paper considers the resource and accuracy trade-off for resource-restricted usages during designing the whole object detection framework. Based on the deeply supervised object detection (DSOD) framework, we propose Tiny-DSOD dedicating to resource-restricted usages. Tiny-D…
Chengdu Institute of Computer Applications, Chinese Academy of Sciences（中国科学院成都计算机应用研究所）；Nanchang Institute of Technology（南昌工程学院）；Zenseact；University of Chinese Academy of Sciences（中国科学院大学）
As one of the prevalent components, Feature Pyramid Network (FPN) is widely used in current object detection models for improving multi-scale object detection performance. However, its feature fusion mode is still in a misaligned and local manner, thus limiting the representation power. To address the inherit defects of FPN, a novel architecture termed Content-Augmented Feature Pyramid Network (CA-FPN) is proposed in this paper. Firstly, a Global Content Extraction Module (GCEM) is proposed to extract multi-scale context information. Secondly, lightweight linear spatial Transformer connections…
Feature pyramid networks (FPN) are widely exploited for multi-scale feature fusion in existing advanced object detection frameworks. Numerous previous works have developed various structures for bidirectional feature fusion, all of which are shown to improve the detection performance effectively. We observe that these complicated network structures require feature pyramids to be stacked in a fixed order, which introduces longer pipelines and reduces the inference speed. Moreover, semantics from non-adjacent levels are diluted in the feature pyramid since only features at adjacent pyramid level…
Beijing University of Posts and Telecommunications（北京邮电大学）；AI Labs, Didi Chuxing（滴滴出行人工智能实验室）
The performance of face detectors has been largely improved with the development of convolutional neural network. However, it remains challenging for face detectors to detect tiny, occluded or blurry faces. Besides, most face detectors can't locate face's position precisely and can't achieve high Intersection-over-Union (IoU) scores. We assume that problems inside are inadequate use of supervision information and imbalance between semantics and details at all level feature maps in CNN even with Feature Pyramid Networks (FPN). In this paper, we present a novel single-shot face detection network…
Semantic-Guided Natural Language and Visual Fusion for Cross-Modal Interaction Based on Tiny Object Detection
Department of Electrical Engineering, National Formosa University, Taiwan（台湾国立Formosa大学电子工程系）；Department of Electrical Engineering, National Taipei University, Taiwan（台湾国立台北大学电子工程系）；College of Artificial Intelligence and Green Energy, National Yang Ming Chiao Tung University, Taiwan（台湾国立阳明交通大学人工智能与再生能源学院）
COXNet: Cross-Layer Fusion with Adaptive Alignment and Scale Integration for RGBT Tiny Object Detection
Beijing Institute of Technology（北京理工大学）；National Astronomical Observatories, Chinese Academy of Sciences（中国科学院国家天文台）；National Key Laboratory of Space Target Awareness, Space Engineering University（航天工程大学空间目标感知全国重点实验室）；Key Laboratory of Photoelectronic Imaging Technology and System, Ministry of Education of China（教育部光电成像技术与系统重点实验室）；Chongqing Innovation Center, Beijing Institute of Technology（北京理工大学重庆创新中心）；China North Vehicle Research Institute（中国北方车辆研究所）
CAS Center for Excellence in Brain Science and Intelligence Technology（中国科学院脑科学与智能技术卓越创新中心）；MEGVII Technology（旷视科技）
This paper revisits feature pyramids networks (FPN) for one-stage detectors and points out that the success of FPN is due to its divide-and-conquer solution to the optimization problem in object detection rather than multi-scale feature fusion. From the perspective of optimization, we introduce an alternative way to address the problem instead of adopting the complex feature pyramids - {\em utilizing only one-level feature for detection}. Based on the simple and efficient solution, we present You Only Look One-level Feature (YOLOF). In our method, two key components, Dilated Encoder and Unifor…
School of Informatics, Xiamen University（厦门大学信息学院）；College of Computer Science and Technology, Huaqiao University（华侨大学计算机科学与技术学院）；University of Luxembourg（卢森堡大学）；School of Computer Science and Technology, Zhejiang Normal University（浙江师范大学计算机科学与技术学院）
Annotating tens or hundreds of tiny objects in a given image is laborious yet crucial for a multitude of Computer Vision tasks. Such imagery typically contains objects from various categories, yet the multi-class interactive annotation setting for the detection task has thus far been unexplored. To address these needs, we propose a novel interactive annotation method for multiple instances of tiny objects from multiple classes, based on a few point-based user inputs. Our approach, C3Det, relates the full image context with annotator inputs in a local and global manner via late-fusion and featu…
MSDet: Receptive Field Enhanced Multiscale Detection for Tiny Pulmonary Nodule
College of Computer Science and Artificial Intelligence, Southwest Minzu University（西南民族大学计算机科学与人工智能学院）；School of Intelligent System Engineering, Sun Yat-sen University（中山大学智能系统工程学院）；University of Warwick（华威大学）；Australian National University（澳大利亚国立大学）；School of Computer Science and Technology, Beijing Jiaotong University（北京交通大学计算机科学与技术学院）；University of Toronto（多伦多大学）；University of Adelaide（阿德莱德大学）；La Trobe University（拉筹伯大学）
With the increasing demand for search and rescue, it is highly demanded to detect objects of interest in large-scale images captured by Unmanned Aerial Vehicles (UAVs), which is quite challenging due to extremely small scales of objects. Most existing methods employed Feature Pyramid Network (FPN) to enrich shallow layers' features by combing deep layers' contextual features. However, under the limitation of the inconsistency in gradient computation across different layers, the shallow layers in FPN are not fully exploited to detect tiny objects. In this paper, we propose a Scale Selection Pyr…
State Key Laboratory for Novel Software Technology, Nanjing University（南京大学计算机软件新技术国家重点实验室）；Megvii Research Nanjing, Megvii Technology（旷视科技南京研究院）
State-of-the-art two-stage object detectors apply a classifier to a sparse set of object proposals, relying on region-wise features extracted by RoIPool or RoIAlign as inputs. The region-wise features, in spite of aligning well with the proposal locations, may still lack the crucial context information which is necessary for filtering out noisy background detections, as well as recognizing objects possessing no distinctive appearances. To address this issue, we present a simple but effective Hierarchical Context Embedding (HCE) framework, which can be applied as a plug-and-play component, to f…
College of Computing, Yonsei University（延世大学计算学院）；CLOVA ImageVision, NAVER（NAVER CLOVA图像视觉部门）
This work is for designing one-stage lightweight detectors which perform well in terms of mAP and latency. With baseline models each of which targets on GPU and CPU respectively, various operations are applied instead of the main operations in backbone networks of baseline models. In addition to experiments about backbone networks and operations, several feature pyramid network (FPN) architectures are investigated. Benchmarks and proposed detectors are analyzed in terms of the number of parameters, Gflops, GPU latency, CPU latency and mAP, on MS COCO dataset which is a benchmark dataset in obj…
CLFTv2: Efficient Camera-LiDAR Fusion for Semantic Segmentation via Hierarchical Feature Pyramids
Tallinn University of Technology（塔林理工大学）；FinEst Centre for Smart Cities, Tallinn University of Technology（塔林理工大学芬埃斯特智慧城市中心）；Universitas Mercatorum（梅尔卡托鲁姆大学）
This paper proposes a novel object detection framework named Grid R-CNN, which adopts a grid guided localization mechanism for accurate object detection. Different from the traditional regression based methods, the Grid R-CNN captures the spatial information explicitly and enjoys the position sensitive property of fully convolutional architecture. Instead of using only two independent points, we design a multi-point supervision formulation to encode more clues in order to reduce the impact of inaccurate prediction of specific points. To take the full advantage of the correlation of points in a…
Beihang University（北京航空航天大学）；Hebei University of Technology（河北工业大学）
The multi-scale defect detection for photovoltaic (PV) cell electroluminescence (EL) images is a challenging task, due to the feature vanishing as network deepens. To address this problem, an attention-based top-down and bottom-up architecture is developed to accomplish multi-scale feature fusion. This architecture, called Bidirectional Attention Feature Pyramid Network (BAFPN), can make all layers of the pyramid share similar semantic features. In BAFPN, cosine similarity is employed to measure the importance of each pixel in the fused features. Furthermore, a novel object detector is propose…
Beijing Institute of Technology（北京理工大学）；Samsung Strategy and Innovation Center（三星战略与创新中心）
For Convolutional Neural Network-based object detection, there is a typical dilemma: the spatial information is well kept in the shallow layers which unfortunately do not have enough semantic information, while the deep layers have a high semantic concept but lost a lot of spatial information, resulting in serious information imbalance. To acquire enough semantic information for shallow layers, Feature Pyramid Networks (FPN) is used to build a top-down propagated path. In this paper, except for top-down combining of information for shallow layers, we propose a novel network called Image Pyrami…
Huazhong University of Science and Technology（华中科技大学）；ETH Zürich（苏黎世联邦理工学院）；Hainan University（海南大学）；Hubei Chutian Expressway Digital Technology（湖北楚天高速数字科技有限公司）
Recent research about camouflaged object detection (COD) aims to segment highly concealed objects hidden in complex surroundings. The tiny, fuzzy camouflaged objects result in visually indistinguishable properties. However, current single-view COD detectors are sensitive to background distractors. Therefore, blurred boundaries and variable shapes of the camouflaged objects are challenging to be fully captured with a single-view detector. To overcome these obstacles, we propose a behavior-inspired framework, called Multi-view Feature Fusion Network (MFFN), which mimics the human behaviors of fi…
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；School of Artificial Intelligence, UCAS（中国科学院大学人工智能学院）；PeopleAI Inc（PeopleAI公司）；Chinese Academy of Sciences（中国科学院）
Although object detection has reached a milestone thanks to the great success of deep learning, the scale variation is still the key challenge. Integrating multi-level features is presented to alleviate the problems, like the classic Feature Pyramid Network (FPN) and its improvements. However, the specifically designed feature integration modules of these methods may not have the optimal architecture for feature fusion. Moreover, these models have fixed architectures and data flow paths, when fed with various samples. They cannot adjust and be compatible with each kind of data. To overcome the…
HCF-Net: Hierarchical Context Fusion Network for Infrared Small Object Detection
Beijing University of Posts and Telecommunications（北京邮电大学）；Shandong Computer Science Center, Qilu University of Technology (Shandong Academy of Sciences)（齐鲁工业大学（山东省科学院）山东计算机科学中心）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；University of Chinese Academy of Sciences（中国科学院大学）；Shandong Fundamental Research Center for Computer Science（山东计算机科学基础研究中心）；Harbin University of Science and Technology（哈尔滨理工大学）
PAN: Pillars-Attention-Based Network for 3D Object Detection
Department of Electronic and Computer Engineering, Lero (the Research Ireland Centre for Software), and the Data Driven Computer Engineering (D 2 iCE) Research Centre at the University of Limerick（电子与计算机工程系，Lero（爱尔兰软件研究中心），以及利默里克大学数据驱动计算机工程（D 2 iCE）研究中心）；Provizio, Future Mobility Campus Ireland, Shannon Free Zone（Provizio，Future Mobility Campus Ireland，Shannon Free Zone）
Beijing Jiaotong University（北京交通大学）；National University of Singapore（新加坡国立大学）
Feature pyramid network (FPN) based models, which fuse the semantics and salient details in a progressive manner, have been proven highly effective in salient object detection. However, it is observed that these models often generate saliency maps with incomplete object structures or unclear object boundaries, due to the \emph{indirect} information propagation among distant layers that makes such fusion structure less effective. In this work, we propose a novel Cross-layer Feature Pyramid Network (CFPN), in which direct cross-layer communication is enabled to improve the progressive fusion in …
LAF-YOLOv10 with Partial Convolution Backbone, Attention-Guided Feature Pyramid, Auxiliary P2 Head, and Wise-IoU Loss for Small Object Detection in Drone Aerial Imagery
Department of Information Technology（信息科技系）；Department of Computer Science and Engineering（计算机科学与工程系）
Wavelet-based Multi-View Fusion of 4D Radar Tensor and Camera for Robust 3D Object Detection
Thrust of Artificial Intelligence, Hong Kong University of Science and Technology (Guangzhou)（人工智能 thrust，香港科技大学（广州））；Momoniai AI；Department of Mechanical Engineering, Massachusetts Institute of Technology（机械工程系，麻省理工学院）；School of Information Engineering, Yancheng Institute of Technology（信息工程学院，盐城科技学院）；College of Information Science and Electronic Engineering, Zhejiang University（信息科学与电子工程学院，浙江大学）；Institute for Math & AI, Wuhan University（数学与人工智能研究所，武汉大学）；College of Science and Engineering and the Centre for AI and Data Science Innovation, James Cook University（科学与工程学院及人工智能与数据科学创新中心，詹姆斯库克大学）；School of Transportation, Southeast University（交通运输学院，东南大学）
Small Object Detection Model with Spatial Laplacian Pyramid Attention and Multi-Scale Features Enhancement in Aerial Images
Key Laboratory of Computational Intelligence and Chinese Information Processing of Ministry of Education, School of Computer and Information Technology, Shanxi University（教育部计算智能与中文信息处理重点实验室，计算机与信息技术学院，山西大学）
Central China Normal University（华中师范大学）；Northwest Normal University（西北师范大学）
Photovoltaic (PV) panel surface-defect detection technology is crucial for the PV industry to perform smart maintenance. Using computer vision technology to detect PV panel surface defects can ensure better accuracy while reducing the workload of traditional worker field inspections. However, multiple tiny defects on the PV panel surface and the high similarity between different defects make it challenging to {accurately identify and detect such defects}. This paper proposes an approach named Ghost convolution with BottleneckCSP and a tiny target prediction head incorporating YOLOv5 (GBH-YOLOv…
