College of Intelligence and Computing, Tianjin University（天津大学智能与计算学部）；Key Research Center for Surface Monitoring and Analysis of Relics, State Administration of Cultural Heritage（国家文物局文物表面监测与分析重点研究中心）；Faculty of Computer Science and Artificial Intelligence, Shenzhen University of Advanced Technology（深圳理工大学计算机科学与人工智能学院）
Yupeng Zhang, Fangzhuo Gao, Ruize Han, Wei Feng, Liang Wan
In real-world deployment under strict real-time constraints, weather and imaging variations induce significant distribution shifts, severely degrading detectors. Single-Domain Generalized Object Detection aims to mitigate this issue, yet existing methods rarely investigate-at the level of problem formulation-the generalization capability of real-time detectors under such constrained inference budgets. To this end, we introduce Real-Time Single-Domain Generalized Object Detection (RT-SDGOD), which focuses on how real-time detectors can achieve cross-domain generalization under zero extra inference overhead by relying solely on training-time representation learning. We observe that, under domain shift, DETR-based real-time detectors mainly degrade through increased missed detections, rooted in limited and unstable object-level discriminative evidence. Based on this, we propose RT-SDGDet, a multi-evidence collaborative modeling framework for RT-SDGOD. The core idea is to enable multiple queries of the same object to collaboratively cover more sufficient discriminative evidence while maintaining the stability of such evidence modeling across views. Specifically, we use one-to-many (O2M) supervision to construct stable object-specific query groups, and further design Discriminative Evidence Diversity Learning (DEDL) and Dual-view Evidence Consistency Learning (DvECL) to expand object-level evidence coverage and improve evidence stability under appearance perturbations, respectively. Since all components are introduced only during training, our method incurs no extra inference overhead. Extensive experiments show that the proposed method achieves better generalization performance than existing approaches across multiple unseen target domains.
VFM$^{4}$SDG: Unveiling the Power of VFMs for Single-Domain Generalized Object Detection
College of Intelligence and Computing, Tianjin University（天津大学智能与计算学院）；Faculty of Computer Science and Artificial Intelligence, Shenzhen University of Advanced Technology（深圳先进技术大学计算机科学与人工智能学院）
Towards Single-Source Domain Generalized Object Detection via Causal Visual Prompts
National Key Laboratory of Multispectral Information Intelligent Processing Technology, School of Artificial Intelligence and Automation, Huazhong University of Science and Technology（multispectral information intelligent processing technology national key laboratory, school of artificial intelligence and automation, huazhong university of science and technology）；Zhejiang Key Laboratory of Intelligent Education Technology and Application, Zhejiang Normal University（intelligent education technology and application zhejiang key laboratory, zhejiang normal university）；Northwest Polytechnical University；Nankai University
GOOD: Towards Domain Generalized Orientated Object Detection
School of Computer Science, Wuhan University（武汉大学计算机学院）；Faculty of Science, University of Amsterdam（阿姆斯特丹大学理学院）；School of Medicine, Yale University（耶鲁大学医学院）；Faculty of Information Technology, Monash University（莫纳什大学信息技术学院）
Recently few-shot object detection is widely adopted to deal with data-limited situations. While most previous works merely focus on the performance on few-shot categories, we claim that detecting all classes is crucial as test samples may contain any instances in realistic applications, which requires the few-shot detector to learn new concepts without forgetting. Through analysis on transfer learning based methods, some neglected but beneficial properties are utilized to design a simple yet effective few-shot detector, Retentive R-CNN. It consists of Bias-Balanced RPN to debias the pretraine…
Robert Bosch GmbH（罗伯特·博世有限公司）；University of Stuttgart（斯图加特大学）；Karlsruhe Institute of Technology（卡尔斯鲁厄理工学院）；Fraunhofer IOSB（弗劳恩霍夫IOSB研究所）
Few-shot object detection (FSOD) seeks to detect novel categories with limited data by leveraging prior knowledge from abundant base data. Generalized few-shot object detection (G-FSOD) aims to tackle FSOD without forgetting previously seen base classes and, thus, accounts for a more realistic scenario, where both classes are encountered during test time. While current FSOD methods suffer from catastrophic forgetting, G-FSOD addresses this limitation yet exhibits a performance drop on novel tasks compared to the state-of-the-art FSOD. In this work, we propose a constraint-based finetuning appr…
Locating 3D objects from a single RGB image via Perspective-n-Points (PnP) is a long-standing problem in computer vision. Driven by end-to-end deep learning, recent studies suggest interpreting PnP as a differentiable layer, so that 2D-3D point correspondences can be partly learned by backpropagating the gradient w.r.t. object pose. Yet, learning the entire set of unrestricted 2D-3D points from scratch fails to converge with existing approaches, since the deterministic pose is inherently non-differentiable. In this paper, we propose the EPro-PnP, a probabilistic PnP layer for general end-to-en…
Improving Single Domain-Generalized Object Detection: A Focus on Diversification and Alignment
Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）；Information Technology University of Punjab（旁遮普信息技术大学）；Mercedes-Benz Tech Innovation（梅赛德斯-奔驰技术创新公司）；Karlsruhe Institute of Technology（卡尔斯鲁厄理工学院）
Boosting Single-domain Generalized Object Detection via Vision-Language Knowledge Interaction
School of Advanced Interdisciplinary Sciences, University of Chinese Academy of Sciences（先进交叉学科研究院，中国科学院大学）；Institute of Microelectronics of the Chinese Academy of Sciences（中国科学院微电子研究所）
Phrase Grounding-based Style Transfer for Single-Domain Generalized Object Detection
School of Computer, National University of Defense Technology, Changsha, China（国防科技大学计算机学院，中国长沙）；School of Cyber Science and Technology, Shenzhen Campus of Sun Yat-sen University, Shenzhen, China（中山大学深圳校区计算机科学与技术学院，中国深圳）；Hong Kong Polytechnic University, Department of Computing, Hong Kong（香港理工大学计算学院，香港）；College of Computer Science and Electronic Engineering, Hunan University, Changsha, China（湖南大学计算机科学与电子工程学院，中国长沙）
National University of Singapore（新加坡国立大学）；SIMTech, Agency for Science, Technology and Research（新加坡科技研究局智能系统研究院(SIMTech)）
Real-world object detection is highly desired to be equipped with the learning expandability that can enlarge its detection classes incrementally. Moreover, such learning from only few annotated training samples further adds the flexibility for the object detector, which is highly expected in many applications such as autonomous driving, robotics, etc. However, such sequential learning scenario with few-shot training samples generally causes catastrophic forgetting and dramatic overfitting. In this paper, to address the above incremental few-shot learning issues, a novel Incremental Few-Shot O…
Towards Domain-Generalized Open-Vocabulary Object Detection: A Progressive Domain-invariant Cross-modal Alignment Method
MAIS, Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所多模态人工智能系统实验室）；Institute of Microelectronics, University of Chinese Academy of Sciences（中国科学院大学微电子学院）；School of Advanced Interdisciplinary Sciences, University of Chinese Academy of Sciences（中国科学院大学前沿交叉科学学院）
Vision-based object detection is one of the fundamental functions in numerous traffic scene applications such as self-driving vehicle systems and advance driver assistance systems (ADAS). However, it is also a challenging task due to the diversity of traffic scene and the storage, power and computing source limitations of the platforms for traffic scene applications. This paper presents a generalized Haar filter based deep network which is suitable for the object detection tasks in traffic scene. In this approach, we first decompose a object detection task into several easier local regression …
Generalized few-shot object detection aims to achieve precise detection on both base classes with abundant annotations and novel classes with limited training data. Existing approaches enhance few-shot generalization with the sacrifice of base-class performance, or maintain high precision in base-class detection with limited improvement in novel-class adaptation. In this paper, we point out the reason is insufficient Discriminative feature learning for all of the classes. As such, we propose a new training framework, DiGeo, to learn Geometry-aware features of inter-class separation and intra-c…
Valeo R&D（法雷奥研发中心）；Valeo Vision Systems（法雷奥视觉系统公司）；Valeo DAR Kronach（法雷奥克罗纳赫DAR）；University of Limerick（利默里克大学）
Object detection is a comprehensively studied problem in autonomous driving. However, it has been relatively less explored in the case of fisheye cameras. The standard bounding box fails in fisheye cameras due to the strong radial distortion, particularly in the image's periphery. We explore better representations like oriented bounding box, ellipse, and generic polygon for object detection in fisheye images in this work. We use the IoU metric to compare these representations using accurate instance segmentation ground truth. We design a novel curved bounding box model that has optimal propert…
Stanford University（斯坦福大学）；The University of Adelaide（阿德莱德大学）；Aibee Inc（Aibee公司）
Intersection over Union (IoU) is the most popular evaluation metric used in the object detection benchmarks. However, there is a gap between optimizing the commonly used distance losses for regressing the parameters of a bounding box and maximizing this metric value. The optimal objective for a metric is the metric itself. In the case of axis-aligned 2D bounding boxes, it can be shown that $IoU$ can be directly used as a regression loss. However, $IoU$ has a plateau making it infeasible to optimize in the case of non-overlapping bounding boxes. In this paper, we address the weaknesses of $IoU$…
Computing a consensus object from a set of given objects is a core problem in machine learning and pattern recognition. One popular approach is to formulate it as an optimization problem using the generalized median. Previous methods like the Prototype and Distance-Preserving Embedding methods transform objects into a vector space, solve the generalized median problem in this space, and inversely transform back into the original space. Both of these methods have been successfully applied to a wide range of object domains, where the generalized median problem has inherent high computational com…
Graz University of Technology（格拉茨理工大学）；Laboratoire Bordelais de Recherche en Informatique（波尔多计算机研究实验室）；Université de Bordeaux（波尔多大学）；X, Alphabet Inc.（Alphabet公司X部门）
We propose an approach to estimating the 3D pose of a hand, possibly handling an object, given a depth image. We show that we can correct the mistakes made by a Convolutional Neural Network trained to predict an estimate of the 3D pose by using a feedback loop. The components of this feedback loop are also Deep Networks, optimized using training data. This approach can be generalized to a hand interacting with an object. Therefore, we jointly estimate the 3D pose of the hand and the 3D pose of the object. Our approach performs en-par with state-of-the-art methods for 3D hand pose estimation, a…
GenHOI: Generalized Hand-Object Pose Estimation with Occlusion Awareness
Hunan University（湖南大学）；Lancaster University（兰卡斯特大学）；University of Western Australia（西澳大学）；University of Trento（特伦托大学）；National University of Singapore（新加坡国立大学）
Beijing Institute of Technology（北京理工大学）；Inception Institute of Artificial Intelligence（Inception人工智能研究院）；Yangtze Delta Region Academy of Beijing Institute of Technology（北京理工大学长三角研究院）；University of Macau（澳门大学）
Recent years have witnessed huge successes in 3D object detection to recognize common objects for autonomous driving (e.g., vehicles and pedestrians). However, most methods rely heavily on a large amount of well-labeled training data. This limits their capability of detecting rare fine-grained objects (e.g., police cars and ambulances), which is important for special cases, such as emergency rescue, and so on. To achieve simultaneous detection for both common and rare objects, we propose a novel task, called generalized few-shot 3D object detection, where we have a large amount of training dat…
U. of Southern California（南加州大学）；U. of Central Florida（中佛罗里达大学）；U. of California, Los Angeles（加州大学洛杉矶分校）
Zero-shot learning (ZSL) methods have been studied in the unrealistic setting where test data are assumed to come from unseen classes only. In this paper, we advocate studying the problem of generalized zero-shot learning (GZSL) where the test data's class memberships are unconstrained. We show empirically that naively using the classifiers constructed by ZSL approaches does not perform well in the generalized setting. Motivated by this, we propose a simple but effective calibration method that can be used to balance two conflicting forces: recognizing data from seen classes versus those from …
University of Illinois at Chicago（伊利诺伊大学芝加哥分校）；Florida State University（佛罗里达州立大学）
Generalized compositional zero-shot learning means to learn composed concepts of attribute-object pairs in a zero-shot fashion, where a model is trained on a set of seen concepts and tested on a combined set of seen and unseen concepts. This task is very challenging because of not only the gap between seen and unseen concepts but also the contextual dependency between attributes and objects. This paper introduces a new approach, termed translational concept embedding, to solve these two difficulties in a unified framework. It models the effect of applying an attribute to an object as adding a …
We propose FindIt, a simple and versatile framework that unifies a variety of visual grounding and localization tasks including referring expression comprehension, text-based localization, and object detection. Key to our architecture is an efficient multi-scale fusion module that unifies the disparate localization requirements across the tasks. In addition, we discover that a standard object detector is surprisingly effective in unifying these tasks without a need for task-specific design, losses, or pre-computed detections. Our end-to-end trainable framework responds flexibly and accurately …
A solution to generalized learning from small training sets found in infants repeated visual experiences of individual objects
Department of Computer Science, Luddy School of Informatics, Computing, and Engineering（计算机科学系，琳迪学校信息学、计算与工程学院）；Department of Psychological and Brain Sciences（心理学与脑科学系）
Simon Fraser University（西蒙菲莎大学）；University of Regina（里贾纳大学）
We present a generalized grasping algorithm that uses point clouds (i.e. a group of points and their respective surface normals) to discover grasp pose solutions for multiple grasp types, executed by a mechanical gripper, in near real-time. The algorithm introduces two ideas: 1) a histogram of finger contact normals is used to represent a grasp 'shape' to guide a gripper orientation search in a histogram of object(s) surface normals, and 2) voxel grid representations of gripper and object(s) are cross-correlated to match finger contact points, i.e. grasp 'size', to discover a grasp pose. Const…
NICTA（国家信息与通信技术研究院）；Australian National University（澳大利亚国立大学）；University of Adelaide（阿德莱德大学）
In this work we generalize the plain MS trackers and attempt to overcome standard mean shift trackers' two limitations. It is well known that modeling and maintaining a representation of a target object is an important component of a successful visual tracker. However, little work has been done on building a robust template model for kernel-based MS tracking. In contrast to building a template from a single frame, we train a robust object representation model from a large amount of data. Tracking is viewed as a binary classification problem, and a discriminative classification rule is learned …
Nanjing University of Science and Technology（南京理工大学）
Zero-shot object recognition or zero-shot learning aims to transfer the object recognition ability among the semantically related categories, such as fine-grained animal or bird species. However, the images of different fine-grained objects tend to merely exhibit subtle differences in appearance, which will severely deteriorate zero-shot object recognition. To reduce the superfluous information in the fine-grained objects, in this paper, we propose to learn the redundancy-free features for generalized zero-shot learning. We achieve our motivation by projecting the original visual features into…
In this Book we argue that the fruitful interaction of computer vision and belief calculus is capable of stimulating significant advances in both fields. From a methodological point of view, novel theoretical results concerning the geometric and algebraic properties of belief functions as mathematical objects are illustrated and discussed in Part II, with a focus on both a perspective 'geometric approach' to uncertainty and an algebraic solution to the issue of conflicting evidence. In Part III we show how these theoretical developments arise from important computer vision problems (such as ar…
A key objective in multi-view learning is to model the information common to multiple parallel views of a class of objects/events to improve downstream learning tasks. In this context, two open research questions remain: How can we model hundreds of views per event? Can we learn robust multi-view embeddings without any knowledge of how these views are acquired? We present a neural method based on multi-view correlation to capture the information shared across a large number of views by subsampling them in a view-agnostic manner during training. To provide an upper bound on the number of views …
