For most of the anchor-based detectors, Intersection over Union(IoU) is widely utilized to assign targets for the anchors during training. However, IoU pays insufficient attention to the closeness of the anchor's center to the truth box's center. This results in two problems: (1) only one anchor is assigned to most of the slender objects which leads to insufficient supervision information for the slender objects during training and the performance on the slender objects is hurt; (2) IoU can not accurately represent the alignment degree between the receptive field of the feature at the anchor's center and the object. Thus during training, some features whose receptive field aligns better with objects are missing while some features whose receptive field aligns worse with objects are adopted. This hurts the localization accuracy of models. To solve these problems, we firstly design Gaussian Guided IoU(GGIoU) which focuses more attention on the closeness of the anchor's center to the truth box's center. Then we propose GGIoU-balanced learning method including GGIoU-guided assignment strategy and GGIoU-balanced localization loss. The method can assign multiple anchors for each slender object and bias the training process to the features well-aligned with objects. Extensive experiments on the popular benchmarks such as PASCAL VOC and MS COCO demonstrate GGIoU-balanced learning can solve the above problems and substantially improve the performance of the object detection model, especially in the localization accuracy.
Huazhong University of Science and Technology（华中科技大学）
Single-stage object detectors have been widely applied in computer vision applications due to their high efficiency. However, we find that the loss functions adopted by single-stage object detectors hurt the localization accuracy seriously. Firstly, the standard cross-entropy loss for classification is independent of the localization task and drives all the positive examples to learn as high classification scores as possible regardless of localization accuracy during training. As a result, there will be many detections that have high classification scores but low IoU or detections that have lo…
Zhejiang University（浙江大学）；Lomonosov Moscow State University（罗蒙诺索夫莫斯科国立大学）；MBZUAI（穆罕默德·本·扎耶德人工智能大学）；IIAI
Most existing point cloud based 3D object detectors focus on the tasks of classification and box regression. However, another bottleneck in this area is achieving an accurate detection confidence for the Non-Maximum Suppression (NMS) post-processing. In this paper, we add a 3D IoU prediction branch to the regular classification and regression branches. The predicted IoU is used as the detection confidence for NMS. In order to obtain a more accurate IoU prediction, we propose a 3D IoU-Net with IoU sensitive feature learning and an IoU alignment operation. To obtain a perspective-invariant predi…
Two-stage deep object detectors generate a set of regions-of-interest (RoI) in the first stage, then, in the second stage, identify objects among the proposed RoIs that sufficiently overlap with a ground truth (GT) box. The second stage is known to suffer from a bias towards RoIs that have low intersection-over-union (IoU) with the associated GT boxes. To address this issue, we first propose a sampling method to generate bounding boxes (BB) that overlap with a given reference box more than a given IoU threshold. Then, we use this BB generation method to develop a positive RoI (pRoI) generator …
Zhejiang University（浙江大学）；Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）；Inception Institute of Artificial Intelligence（Inception人工智能研究院）
In this paper, we present an Intersection-over-Union (IoU) guided two-stage 3D object detector with a voxel-to-point decoder. To preserve the necessary information from all raw points and maintain the high box recall in voxel based Region Proposal Network (RPN), we propose a residual voxel-to-point decoder to extract the point features in addition to the map-view features from the voxel based RPN. We use a 3D Region of Interest (RoI) alignment to crop and align the features with the proposal boxes for accurately perceiving the object position. The RoI-Aligned features are finally aggregated wi…
UIUC（伊利诺伊大学厄巴纳-香槟分校）；Facebook AI Research (FAIR)（Facebook人工智能研究院）
We present Boundary IoU (Intersection-over-Union), a new segmentation evaluation measure focused on boundary quality. We perform an extensive analysis across different error types and object sizes and show that Boundary IoU is significantly more sensitive than the standard Mask IoU measure to boundary errors for large objects and does not over-penalize errors on smaller objects. The new quality measure displays several desirable characteristics like symmetry w.r.t. prediction/ground truth pairs and balanced responsiveness across scales, which makes it more suitable for segmentation evaluation …
Classification and regression are two pillars of object detectors. In most CNN-based detectors, these two pillars are optimized independently. Without direct interactions between them, the classification loss and the regression loss can not be optimized synchronously toward the optimal direction in the training phase. This clearly leads to lots of inconsistent predictions with high classification score but low localization accuracy or low classification score but high localization accuracy in the inference phase, especially for the objects of irregular shape and occlusion, which severely hurts…
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；University of Chinese Academy of Sciences（中国科学院大学）；Chinese Academy of Sciences（中国科学院）
Based on multiple instance detection networks (MIDN), plenty of works have contributed tremendous efforts to weakly supervised object detection (WSOD). However, most methods neglect the fact that the overwhelming negative instances exist in each image during the training phase, which would mislead the training and make the network fall into local minima. To tackle this problem, an online progressive instance-balanced sampling (OPIS) algorithm based on hard sampling and soft sampling is proposed in this paper. The algorithm includes two modules: a progressive instance balance (PIB) module and a…
Zhejiang University（浙江大学）；The Chinese University of Hong Kong（香港中文大学）；SenseTime Research（商汤科技研究院）；The University of Sydney（悉尼大学）
Compared with model architectures, the training process, which is also crucial to the success of detectors, has received relatively less attention in object detection. In this work, we carefully revisit the standard training practice of detectors, and find that the detection performance is often limited by the imbalance during the training process, which generally consists in three levels - sample level, feature level, and objective level. To mitigate the adverse effects caused thereby, we propose Libra R-CNN, a simple but effective framework towards balanced learning for object detection. It …
LAF-YOLOv10 with Partial Convolution Backbone, Attention-Guided Feature Pyramid, Auxiliary P2 Head, and Wise-IoU Loss for Small Object Detection in Drone Aerial Imagery
Department of Information Technology（信息科技系）；Department of Computer Science and Engineering（计算机科学与工程系）
This paper proposes a novel object detection framework named Grid R-CNN, which adopts a grid guided localization mechanism for accurate object detection. Different from the traditional regression based methods, the Grid R-CNN captures the spatial information explicitly and enjoys the position sensitive property of fully convolutional architecture. Instead of using only two independent points, we design a multi-point supervision formulation to encode more clues in order to reduce the impact of inaccurate prediction of specific points. To take the full advantage of the correlation of points in a…
Mainstream object detectors are commonly constituted of two sub-tasks, including classification and regression tasks, implemented by two parallel heads. This classic design paradigm inevitably leads to inconsistent spatial distributions between classification score and localization quality (IOU). Therefore, this paper alleviates this misalignment in the view of knowledge distillation. First, we observe that the massive teacher achieves a higher proportion of harmonious predictions than the lightweight student. Based on this intriguing observation, a novel Harmony Score (HS) is devised to estim…
Università degli studi di Milano（米兰大学）；Fondazione IRCCS Ca’ Granda, Ospedale Maggiore Policlinico di Milano（米兰马焦雷医院格兰达卡基金会IRCCS）
Joint bleeding is a common condition for people with hemophilia and, if untreated, can result in hemophilic arthropathy. Ultrasound imaging has recently emerged as an effective tool to diagnose joint recess distension caused by joint bleeding. However, no computer-aided diagnosis tool exists to support the practitioner in the diagnosis process. This paper addresses the problem of automatically detecting the recess and assessing whether it is distended in knee ultrasound images collected in patients with hemophilia. After framing the problem, we propose two different approaches: the first one a…
Most deep learning object detectors are based on the anchor mechanism and resort to the Intersection over Union (IoU) between predefined anchor boxes and ground truth boxes to evaluate the matching quality between anchors and objects. In this paper, we question this use of IoU and propose a new anchor matching criterion guided, during the training phase, by the optimization of both the localization and the classification tasks: the predictions related to one task are used to dynamically assign sample anchors and improve the model on the other task, and vice versa. Despite the simplicity of the…
Edge-guided and Class-balanced Active Learning for Semantic Segmentation of Aerial Images
School of Computer Science and Technology, University of Chinese Academy of Sciences（中国科学院大学计算机科学与技术学院）；School of Engineering Science, University of Chinese Academy of Sciences（中国科学院大学工程科学学院）；School of Computer Science and Technology, Anhui University（安徽大学计算机科学与技术学院）
Sun Yat-sen University（中山大学）；Guangdong Province Key Laboratory of Information Security Technology（广东省信息安全技术重点实验室）；Key Laboratory of Machine Intelligence and Advanced Computing, Ministry of Education（教育部机器智能与先进计算重点实验室）
Video salient object detection (VSOD) aims to locate and segment the most attractive object by exploiting both spatial cues and temporal cues hidden in video sequences. However, spatial and temporal cues are often unreliable in real-world scenarios, such as low-contrast foreground, fast motion, and multiple moving objects. To address these problems, we propose a new framework to adaptively capture available information from spatial and temporal cues, which contains Confidence-guided Adaptive Gate (CAG) modules and Dual Differential Enhancement (DDE) modules. For both RGB features and optical f…
Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）；University of Chinese Academy of Sciences（中国科学院大学）
The demand of applying semantic segmentation model on mobile devices has been increasing rapidly. Current state-of-the-art networks have enormous amount of parameters hence unsuitable for mobile devices, while other small memory footprint models follow the spirit of classification network and ignore the inherent characteristic of semantic segmentation. To tackle this problem, we propose a novel Context Guided Network (CGNet), which is a light-weight and efficient network for semantic segmentation. We first propose the Context Guided (CG) block, which learns the joint feature of both local feat…
Barcelona Supercomputing Center（巴塞罗那超级计算中心）；Universitat Politècnica de Catalunya（加泰罗尼亚理工大学）
Image segmentation methods are usually trained with pixel-level annotations, which require significant human effort to collect. The most common solution to address this constraint is to implement weakly-supervised pipelines trained with lower forms of supervision, such as bounding boxes or scribbles. Another option are semi-supervised methods, which leverage a large amount of unlabeled data and a limited number of strongly-labeled samples. In this second setup, samples to be strongly-annotated can be selected randomly or with an active learning mechanism that chooses the ones that will maximiz…
Beijing Institute of Technology（北京理工大学）；Aerospace Information Research Institute, Chinese Academy of Sciences（中国科学院空天信息创新研究院）；Hubei University of Economics（湖北经济学院）；Luoyang Institute of Electro-Optical Equipment, Aviation Industry Corporation of China, Ltd.（中国航空工业集团有限公司洛阳电光设备研究所）
Arbitrary-oriented object detection (AOOD) plays a significant role for image understanding in remote sensing scenarios. The existing AOOD methods face the challenges of ambiguity and high costs in angle representation. To this end, a multi-grained angle representation (MGAR) method, consisting of coarse-grained angle classification (CAC) and fine-grained angle regression (FAR), is proposed. Specifically, the designed CAC avoids the ambiguity of angle prediction by discrete angular encoding (DAE) and reduces complexity by coarsening the granularity of DAE. Based on CAC, FAR is developed to ref…
Bridging the Scale Gap: Balanced Tiny and General Object Detection in Remote Sensing Imagery
Key Laboratory of Intelligent Computing & Signal Processing (Anhui University), Ministry of Education（智能计算与信号处理重点实验室（安徽大学））；Anhui Provincial Key Laboratory of Multimodal Cognitive Computation（安徽省多模态认知计算重点实验室）；School of Artificial Intelligence, Anhui University（人工智能学院）；th Research Institute, China Electronics Technology Group Corporation（中国电子科技集团第三十八研究所）；School of Computer Science and Technology, Anhui University（计算机科学与技术学院）
Detecting tiny objects is one of the main obstacles hindering the development of object detection. The performance of generic object detectors tends to drastically deteriorate on tiny object detection tasks. In this paper, we point out that either box prior in the anchor-based detector or point prior in the anchor-free detector is sub-optimal for tiny objects. Our key observation is that the current anchor-based or anchor-free label assignment paradigms will incur many outlier tiny-sized ground truth samples, leading to detectors imposing less focus on the tiny objects. To this end, we propose…
University of Science and Technology Beijing（北京科技大学）；University of Central Florida（中佛罗里达大学）；Scoop Medical（Scoop医疗）
The encoder-decoder architecture is widely used as a lightweight semantic segmentation network. However, it struggles with a limited performance compared to a well-designed Dilated-FCN model for two major problems. First, commonly used upsampling methods in the decoder such as interpolation and deconvolution suffer from a local receptive field, unable to encode global contexts. Second, low-level features may bring noises to the network decoder through skip connections for the inadequacy of semantic concepts in early encoder layers. To tackle these challenges, a Global Enhancement Method is pro…
This paper presents the novel idea of generating object proposals by leveraging temporal information for video object detection. The feature aggregation in modern region-based video object detectors heavily relies on learned proposals generated from a single-frame RPN. This imminently introduces additional components like NMS and produces unreliable proposals on low-quality frames. To tackle these restrictions, we present SparseVOD, a novel video object detection pipeline that employs Sparse R-CNN to exploit temporal information. In particular, we introduce two modules in the dynamic head of S…
Huazhong University of Science and Technology（华中科技大学）；The University of Adelaide（阿德莱德大学）；The Chinese University of Hong Kong（香港中文大学）；Tencent（腾讯）
The low-level details and high-level semantics are both essential to the semantic segmentation task. However, to speed up the model inference, current approaches almost always sacrifice the low-level details, which leads to a considerable accuracy decrease. We propose to treat these spatial details and categorical semantics separately to achieve high accuracy and high efficiency for realtime semantic segmentation. To this end, we propose an efficient and effective architecture with a good trade-off between speed and accuracy, termed Bilateral Segmentation Network (BiSeNet V2). This architectur…
The Chinese University of Hong Kong（香港中文大学）；SmartMore（思谋科技）
Rapid progress in 3D semantic segmentation is inseparable from the advances of deep network models, which highly rely on large-scale annotated data for training. To address the high cost and challenges of 3D point-level labeling, we present a method for semi-supervised point cloud semantic segmentation to adopt unlabeled point clouds in training to boost the model performance. Inspired by the recent contrastive loss in self-supervised tasks, we propose the guided point contrastive loss to enhance the feature representation and model generalization ability in semi-supervised setting. Semantic p…
Karlsruhe Institute of Technology（卡尔斯鲁厄理工学院）；University Medicine Essen（埃森大学医学院）
Producing densely annotated data is a difficult and tedious task for medical imaging applications. To address this problem, we propose a novel approach to generate supervision for semi-supervised semantic segmentation. We argue that visually similar regions between labeled and unlabeled images likely contain the same semantics and therefore should share their label. Following this thought, we use a small number of labeled images as reference material and match pixels in an unlabeled image to the semantics of the best fitting pixel in a reference set. This way, we avoid pitfalls such as confirm…
University of Alberta（阿尔伯塔大学）；Yazd University（亚兹德大学）；Sharif University of Technology（谢里夫理工大学）
We consider the problem of tracking an unknown small target from aerial videos of medium to high altitudes. This is a challenging problem, which is even more pronounced in unavoidable scenarios of drastic camera motion and high density. To address this problem, we introduce a context-aware IoU-guided tracker (COMET) that exploits a multitask two-stream network and an offline reference proposal generation strategy. The proposed network fully exploits target-related information by multi-scale feature learning and attention modules. The proposed strategy introduces an efficient sampling strategy …
Friedrich-Alexander-Universität Erlangen-Nürnberg（弗里德里希-亚历山大-埃尔兰根-纽伦堡大学）；Siemens Healthcare GmbH（西门子医疗保健有限公司）；Hochschule Furtwangen（富特旺根应用技术大学）；BG Trauma Center Ludwigshafen（路德维希港BG创伤中心）
X-ray based measurement and guidance are commonly used tools in orthopaedic surgery to facilitate a minimally invasive workflow. Typically, a surgical planning is first performed using knowledge of bone morphology and anatomical landmarks. Information about bone location then serves as a prior for registration during overlay of the planning on intra-operative X-ray images. Performing these steps manually however is prone to intra-rater/inter-rater variability and increases task complexity for the surgeon. To remedy these issues, we propose an automatic framework for planning and subsequent ove…
Real-Time Brain Tumor Detection in Intraoperative Ultrasound Using YOLO11: From Model Training to Deployment in the Operating Room
Department of Neurosurgery, Rio Hortega University Hospital（里奥霍尔特加大学医院神经外科部）；Department of Neurosurgery, Tata Memorial Centre, Homi Bhabha National Institute（塔塔纪念中心神经外科部，霍米·巴布哈国家研究所）；Biomedical Engineering Group, University of Valladolid（瓦拉多利德大学生物医学工程小组）；UMR 1253, iBrain, Université de Tours, Inserm（UMR 1253，iBrain， Tours大学，Inserm）；Department of Neurosurgery, CHRU de Tours（图尔CHRU医院神经外科部）；Department of Neurosrugery, ARNAS Civico Di Cristina Benfratelli Hospital（ARNAS公民迪克里斯塔弗拉蒂医院神经外科部）；Fondazione IRCCS Istituto Neurologico Carlo Besta（IRCCS神经学研究所卡尔罗·贝斯塔基金会）；University of Milan（米兰大学）；Johns Hopkins Medical School（约翰霍普金斯医学院）
Layer-wise Noise Guided Selective Wavelet Reconstruction for Robust Medical Image Segmentation
Big Data and Software College, Chongqing University（重庆大学大数据与软件学院）；Office of Scientific Research, Peking University（北京大学科研办公室）；Key Lab of High Confidence Software Technology, Ministry of Education (Peking University)（教育部高可信软件技术重点实验室（北京大学））；School of Computer Science, Peking University（北京大学计算机学院）；Department of Cardiology and Institute of Vascular Medicine, Peking University Third Hospital（北京大学第三医院心内科和血管医学研究院）
Image guidance has been widely used in radiation therapy. Correctly identifying the bounding box of the anatomical landmarks from limited field of views is the key to success. In image-guided radiation therapy (IGRT), the detection of those landmarks like the 12th vertebra (T12) still requires tedious manual inspections and annotations; and superior-inferior misalignment to the wrong vertebral body is still relatively common. It is necessary to develop an automated approach to detect those landmarks from images. The challenges of training a model to identify T12 vertebrae automatically mainly …
Zhejiang University（浙江大学）；The Chinese University of Hong Kong（香港中文大学）；SenseTime Research（商汤科技研究院）；The University of Sydney（悉尼大学）
Instance recognition is rapidly advanced along with the developments of various deep convolutional neural networks. Compared to the architectures of networks, the training process, which is also crucial to the success of detectors, has received relatively less attention. In this work, we carefully revisit the standard training practice of detectors, and find that the detection performance is often limited by the imbalance during the training process, which generally consists in three levels - sample level, feature level, and objective level. To mitigate the adverse effects caused thereby, we p…
Balanced Diffusion-Guided Fusion for Multimodal Remote Sensing Classification
Department of Information Engineering and Computer Science, University of Trento（特伦托大学信息工程与计算机科学系）；College of Electrical and Information Engineering, Hunan University（湖南大学电气与信息工程学院）；Key Laboratory of Collaborative Intelligent Systems of Ministry of Education, Xidian University（西安电子科技大学教育部协同智能系统重点实验室）；School of Electronic Engineering, Xidian University（西安电子科技大学电子工程学院）；Academy of Artificial Intelligence, Inner Mongolia Normal University（内蒙古师范大学人工智能学院）
Uncertainty-Guided Adverse Weather Restoration via Gated Transformer Network
School of Automation, Southeast University（东南大学自动化学院）；Chair of Robotics, Artificial Intelligence and Real-time Systems, Technical University of Munich（慕尼黑工业大学机器人、人工智能与实时系统讲席教授组）
