Deep neural networks have been demonstrated to be vulnerable to adversarial attacks: subtle perturbations can completely change the classification results. Their vulnerability has led to a surge of research in this direction. However, most works dedicated to attacking anchor-based object detection models. In this work, we aim to present an effective and efficient algorithm to generate adversarial examples to attack anchor-free object models based on two approaches. First, we conduct category-wise instead of instance-wise attacks on the object detectors. Second, we leverage the high-level semantic information to generate the adversarial examples. Surprisingly, the generated adversarial examples it not only able to effectively attack the targeted anchor-free object detector but also to be transferred to attack other object detectors, even anchor-based detectors such as Faster R-CNN.
Chengdu University of Information Technology（成都信息工程大学）；Keya Medical（科亚医疗）；University at Buffalo, State University of New York（纽约州立大学布法罗分校）；Microsoft Research Asia（微软亚洲研究院）
Deep neural networks have been demonstrated to be vulnerable to adversarial attacks: subtle perturbation can completely change prediction result. The vulnerability has led to a surge of research in this direction, including adversarial attacks on object detection networks. However, previous studies are dedicated to attacking anchor-based object detectors. In this paper, we present the first adversarial attack on anchor-free object detectors. It conducts category-wise, instead of previously instance-wise, attacks on object detectors, and leverages high-level semantic information to efficiently …
Almost all current adversarial attacks of CNN classifiers rely on information derived from the output layer of the network. This work presents a new adversarial attack based on the modeling and exploitation of class-wise and layer-wise deep feature distributions. We achieve state-of-the-art targeted blackbox transfer-based attack results for undefended ImageNet models. Further, we place a priority on explainability and interpretability of the attacking process. Our methodology affords an analysis of how adversarial attacks change the intermediate feature distributions of CNNs, as well as a mea…
Chengdu University of Information Technology（成都信息工程大学）；Carnegie Mellon University（卡内基梅隆大学）；University at Buffalo, SUNY（纽约州立大学布法罗分校）；Microsoft Research Asia（微软亚洲研究院）
Deep neural networks have been demonstrated to be vulnerable to adversarial attacks: subtle perturbation can completely change the prediction result. Existing adversarial attacks on object detection focus on attacking anchor-based detectors, which may not work well for anchor-free detectors. In this paper, we propose the first adversarial attack dedicated to anchor-free detectors. It is a category-wise attack that attacks important pixels of all instances of a category simultaneously. Our attack manifests in two forms, sparse category-wise attack (SCA) and dense category-wise attack (DCA), tha…
The anchor-based detectors handle the problem of scale variation by building the feature pyramid and directly setting different scales of anchors on each cell in different layers. However, it is difficult for box-wise anchors to guide the adaptive learning of scale-specific features in each layer because there is no one-to-one correspondence between box-wise anchors and pixel-level features. In order to alleviate the problem, in this paper, we propose a scale-customized weak segmentation (SCWS) block at the pixel level for scale customized object feature learning in each layer. By integrating …
One Attack to Fool Them All: Highly Transferable Black-Box Adversarial Attacks on Frontier MLLMs
State Key Laboratory of AI Safety, Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所人工智能安全国家重点实验室）；University of Chinese Academy of Sciences（中国科学院大学）
Bounding-box regression is a popular technique to refine or predict localization boxes in recent object detection approaches. Typically, bounding-box regressors are trained to regress from either region proposals or fixed anchor boxes to nearby bounding boxes of a pre-defined target object classes. This paper investigates whether the technique is generalizable to unseen classes and is transferable to other tasks beyond supervised object detection. To this end, we propose a class-agnostic and anchor-free box regressor, dubbed Universal Bounding-Box Regressor (UBBR), which predicts a bounding bo…
Robert Bosch GmbH（罗伯特·博世有限公司）；Karlsruhe Institute of Technology（卡尔斯鲁厄理工学院）；University of Stuttgart（斯图加特大学）；Fraunhofer IOSB（弗劳恩霍夫光学、电子与精密工程研究所）
Recent object detection models require large amounts of annotated data for training a new classes of objects. Few-shot object detection (FSOD) aims to address this problem by learning novel classes given only a few samples. While competitive results have been achieved using two-stage FSOD detectors, typically one-stage FSODs underperform compared to them. We make the observation that the large gap in performance between two-stage and one-stage FSODs are mainly due to their weak discriminability, which is explained by a small post-fusion receptive field and a small number of foreground samples …
Most existing object detectors suffer from class imbalance problems that hinder balanced performance. In particular, anchor free object detectors have to solve the background imbalance problem due to detection in a per-pixel prediction fashion as well as foreground imbalance problem simultaneously. In this work, we propose Balance-oriented focal loss that can induce balanced learning by considering both background and foreground balance comprehensively. This work aims to address imbalance problem in the situation of using a general unbalanced data of non-extreme distribution not including few …
The University of Sydney（悉尼大学）；Tencent AI Lab（腾讯人工智能实验室）
Unsupervised domain adaptation (UDA) aims to enhance the generalization capability of a certain model from a source domain to a target domain. UDA is of particular significance since no extra effort is devoted to annotating target domain samples. However, the different data distributions in the two domains, or \emph{domain shift/discrepancy}, inevitably compromise the UDA performance. Although there has been a progress in matching the marginal distributions between two domains, the classifier favors the source domain features and makes incorrect predictions on the target domain due to category…
SegAssess: Panoramic quality mapping for robust and transferable unsupervised segmentation assessment
School of Remote Sensing and Information Engineering, Wuhan University（武汉大学遥感与信息工程学院）；College of Electronic Science and Technology, National University of Defense Technology（国防科技大学电子科学学院）；Hubei Luojia Laboratory（湖北珞珈实验室）
Alibaba Group（阿里巴巴集团）；The Hong Kong Polytechnic University（香港理工大学）
Semi-supervised object detection (SSOD) aims to facilitate the training and deployment of object detectors with the help of a large amount of unlabeled data. Though various self-training based and consistency-regularization based SSOD methods have been proposed, most of them are anchor-based detectors, ignoring the fact that in many real-world applications anchor-free detectors are more demanded. In this paper, we intend to bridge this gap and propose a DenSe Learning (DSL) based anchor-free SSOD algorithm. Specifically, we achieve this goal by introducing several novel techniques, including a…
College of Information Science and Electronic Engineering, Zhejiang University（浙江大学信息科学与电子工程学院）；Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）；Inception Institute of Artificial Intelligence（Inception人工智能研究院）；School of Micro-Nano Electronics, Zhejiang University（浙江大学微纳电子学院）
In this paper, we propose a Monocular 3D Single Stage object Detector (M3DSSD) with feature alignment and asymmetric non-local attention. Current anchor-based monocular 3D object detection methods suffer from feature mismatching. To overcome this, we propose a two-step feature alignment approach. In the first step, the shape alignment is performed to enable the receptive field of the feature map to focus on the pre-defined anchors with high confidence scores. In the second step, the center alignment is used to align the features at 2D/3D centers. Further, it is often difficult to learn global …
In this paper, we present a real-time approach to predict multiple grasping poses for a parallel-plate robotic gripper using RGB images. A model with oriented anchor box mechanism is proposed and a new matching strategy is used during the training process. An end-to-end fully convolutional neural network is employed in our work. The network consists of two parts: the feature extractor and multi-grasp predictor. The feature extractor is a deep convolutional neural network. The multi-grasp predictor regresses grasp rectangles from predefined oriented rectangles, called oriented anchor boxes, and…
PandaPose: 3D Human Pose Lifting from a Single Image via Propagating 2D Pose Prior to 3D Anchor Space
School of Journalism and Information Communication, Huazhong University of Science and Technology（华中科技大学新闻与信息传播学院）；ByteDance Inc.（字节跳动公司）；Institute of High Performance Computing, Agency for Science, Technology and Research, Singapore（科技研究局高性能计算研究所）
State Key Laboratory of Robotics, Shenyang Institute of Automation, Chinese Academy of Sciences（中国科学院沈阳自动化研究所机器人学国家重点实验室）；Institutes for Robotics and Intelligent Manufacturing, Chinese Academy of Sciences（中国科学院机器人与智能制造创新研究院）；University of Chinese Academy of Sciences（中国科学院大学）；University of Arkansas at Little Rock（阿肯色大学小石城分校）
Unsupervised domain adaptation without consuming annotation process for unlabeled target data attracts appealing interests in semantic segmentation. However, 1) existing methods neglect that not all semantic representations across domains are transferable, which cripples domain-wise transfer with untransferable knowledge; 2) they fail to narrow category-wise distribution shift due to category-agnostic feature alignment. To address above challenges, we develop a new Critical Semantic-Consistent Learning (CSCL) model, which mitigates the discrepancy of both domain-wise and category-wise distribu…
SGHA-Attack: Semantic-Guided Hierarchical Alignment for Transferable Targeted Attacks on Vision-Language Models
Guangdong Province Key Lab of Information Security Technology, and School of Computer Science and Engineering, Sun Yat-sen University（广东信息安全技术重点实验室，计算机科学与工程学院，中山大学）；Nanyang Technological University（南洋理工大学）；School of Cyber Science and Technology, Shenzhen Campus, Sun Yat-sen University（网络安全科学与技术学院，深圳校区，中山大学）
Microsoft（微软）；Geroge Washington University（乔治·华盛顿大学）
Pair-wise loss is an approach to metric learning that learns a semantic embedding by optimizing a loss function that encourages images from the same semantic class to be mapped closer than images from different classes. The literature reports a large and growing set of variations of the pair-wise loss strategies. Here we decompose the gradient of these loss functions into components that relate to how they push the relative feature positions of the anchor-positive and anchor-negative pairs. This decomposition allows the unification of a large collection of current pair-wise loss functions. Add…
Contrastive learning based on instance discrimination trains model to discriminate different transformations of the anchor sample from other samples, which does not consider the semantic similarity among samples. This paper proposes a new kind of contrastive learning method, named CLIM, which uses positives from other samples in the dataset. This is achieved by searching local similar samples of the anchor, and selecting samples that are closer to the corresponding cluster center, which we denote as center-wise local image selection. The selected samples are instantiated via an data mixture st…
Academia Sinica（中央研究院）；National Taiwan University（台湾大学）；NEC Laboratories America（美国电气股份有限公司实验室）；University of California, Merced（加州大学默塞德分校）；Google Cloud（谷歌云）
In order to learn object segmentation models in videos, conventional methods require a large amount of pixel-wise ground truth annotations. However, collecting such supervised data is time-consuming and labor-intensive. In this paper, we exploit existing annotations in source images and transfer such visual information to segment videos with unseen object categories. Without using any annotations in the target video, we propose a method to jointly mine useful segments and learn feature representations that better adapt to the target frames. The entire process is decomposed into two tasks: 1) s…
When Robots Obey the Patch: Universal Transferable Patch Attacks on Vision-Language-Action Models
ROSE Lab, Interdisciplinary Graduate Programme, Nanyang Technological University（罗思实验室，跨学科研究生项目，南洋理工大学）；ROSE Lab, School of Electrical and Electronic Engineering, Nanyang Technological University（罗思实验室，电子与电气工程学院，南洋理工大学）；CCDS, Nanyang Technological University（CCDS，南洋理工大学）；DSO National Laboratories（国防科学局实验室）
Zhejiang University（浙江大学）；Shanghai Institute for Advanced Study, Zhejiang University（浙江大学上海高等研究院）；Zhejiang–Singapore Innovation and AI Joint Research Lab（浙江-新加坡创新与人工智能联合实验室）
Modern methods mainly regard lane detection as a problem of pixel-wise segmentation, which is struggling to address the problems of efficiency and challenging scenarios like severe occlusions and extreme lighting conditions. Inspired by human perception, the recognition of lanes under severe occlusions and extreme lighting conditions is mainly based on contextual and global information. Motivated by this observation, we propose a novel, simple, yet effective formulation aiming at ultra fast speed and the problem of challenging scenarios. Specifically, we treat the process of lane detection as …
School of Artificial Intelligence, Xidian University（西安电子科技大学人工智能学院）
Ship detection in aerial images remains an active yet challenging task due to arbitrary object orientation and complex background from a bird's-eye perspective. Most of the existing methods rely on angular prediction or predefined anchor boxes, making these methods highly sensitive to unstable angular regression and excessive hyper-parameter setting. To address these issues, we replace the angular-based object encoding with an anchor-and-angle-free paradigm, and propose a novel detector deploying a center and four midpoints for encoding each oriented object, namely MidNet. MidNet designs a sym…
Proxy Robustness in Vision Language Models is Effortlessly Transferable
Chongqing Key Laboratory of Bio-perception（重庆生物感知实验室）；Multimodal Intelligent Information Processing, Chongqing University, Chongqing 401331, China（多模态智能信息处理，重庆大学，重庆401331，中国）；School of Microelectronics（微电子学院）；Communication Engineering, Chongqing University, Chongqing 401331, China（通信工程，重庆大学，重庆401331，中国）；School of Data Science, Lingnan University, Hong Kong（数据科学学院，岭南大学，香港）
Accurate 6D object pose estimation is fundamental to robotic manipulation and grasping. Previous methods follow a local optimization approach which minimizes the distance between closest point pairs to handle the rotation ambiguity of symmetric objects. In this work, we propose a novel discrete-continuous formulation for rotation regression to resolve this local-optimum problem. We uniformly sample rotation anchors in SO(3), and predict a constrained deviation from each anchor to the target, as well as uncertainty scores for selecting the best prediction. Additionally, the object location is d…
Northeastern University（东北大学）；Changsha Hisense Intelligent System Research Institute Co., Ltd.（长沙海信智能系统研究院有限公司）；Peking University（北京大学）
Human-Object Interaction (HOI), as an important problem in computer vision, requires locating the human-object pair and identifying the interactive relationships between them. The HOI instance has a greater span in spatial, scale, and task than the individual object instance, making its detection more susceptible to noisy backgrounds. To alleviate the disturbance of noisy backgrounds on HOI detection, it is necessary to consider the input image information to generate fine-grained anchors which are then leveraged to guide the detection of HOI instances. However, it is challenging for the follo…
SAM Encoder Breach by Adversarial Simplicial Complex Triggers Downstream Model Failures
State Key Laboratory of Advanced Rail Autonomous Operation（先进轨道交通自主运行状态关键实验室）；Beijing Key Lab of Traffic Data Mining and Embodied Intelligence（北京交通数据挖掘与具身智能重点实验室）；Collaborative Innovation Center of Railway Traffic Safety（铁路交通安全协同创新中心）；National Engineering Research Center of Rail Transportation Operation and Control System（铁路运输操作与控制系统国家工程研究中心）；School of Computer Science and Technology, Beijing Jiaotong University（北京交通大学计算机科学与技术学院）；School of Automation and Intelligence, Beijing Jiaotong University（北京交通大学自动化与智能学院）
Prior state-of-the-art adversarial detection works are classifier model dependent, i.e., they require classifier model outputs and parameters for training the detector or during adversarial detection. This makes their detection approach classifier model specific. Furthermore, classifier model outputs and parameters might not always be accessible. To this end, we propose a classifier model independent adversarial detection method using a simple energy function to distinguish between adversarial and natural inputs. We train a standalone detector independent of the classifier model, with a layer-…
Decompose and Transfer: CoT-Prompting Enhanced Alignment for Open-Vocabulary Temporal Action Detection
Institute of Information Engineering, Chinese Academy of Sciences（中国科学院信息工程研究所）；School of Cyber Security, University of Chinese Academy of Sciences（中国科学院大学网络空间安全学院）；State Key Laboratory of Cyberspace Security Defense（网络空间安全防御国家重点实验室）；Hangzhou Dianzi University（杭州电子科技大学）；Department of Automation, Tsinghua University（清华大学自动化系）
Rotation invariance has been an important topic in computer vision tasks. Ideally, robot grasp detection should be rotation-invariant. However, rotation-invariance in robotic grasp detection has been only recently studied by using rotation anchor box that are often time-consuming and unreliable for multiple objects. In this paper, we propose a rotation ensemble module (REM) for robotic grasp detection using convolutions that rotates network weights. Our proposed REM was able to outperform current state-of-the-art methods by achieving up to 99.2% (image-wise), 98.6% (object-wise) accuracies on …
