In this paper, we focus on category-level 6D pose and size estimation from monocular RGB-D image. Previous methods suffer from inefficient category-level pose feature extraction which leads to low accuracy and inference speed. To tackle this problem, we propose a fast shape-based network (FS-Net) with efficient category-level feature extraction for 6D pose estimation. First, we design an orientation aware autoencoder with 3D graph convolution for latent feature extraction. The learned latent feature is insensitive to point shift and object size thanks to the shift and scale-invariance properties of the 3D graph convolution. Then, to efficiently decode category-level rotation information from the latent feature, we propose a novel decoupled rotation mechanism that employs two decoders to complementarily access the rotation information. Meanwhile, we estimate translation and size by two residuals, which are the difference between the mean of object points and ground truth translation, and the difference between the mean size of the category and ground truth size, respectively. Finally, to increase the generalization ability of FS-Net, we propose an online box-cage based 3D deformation mechanism to augment the training data. Extensive experiments on two benchmark datasets show that the proposed method achieves state-of-the-art performance in both category- and instance-level 6D object pose estimation. Especially in category-level pose estimation, without extra synthetic data, our method outperforms existing methods by 6.3% on the NOCS-REAL dataset.
Politecnico di Torino（都灵理工大学）；Italian Institute of Technology（意大利理工学院）
In this paper, we present a deep learning model that exploits the power of self-supervision to perform 3D point cloud completion, estimating the missing part and a context region around it. Local and global information are encoded in a combined embedding. A denoising pretext task provides the network with the needed local cues, decoupled from the high-level semantics and naturally shared over multiple classes. On the other hand, contrastive learning maximizes the agreement between variants of the same shape with different missing portions, thus producing a representation which captures the glo…
Correlation-Decoupled Knowledge Distillation for Multimodal Sentiment Analysis with Incomplete Modalities
Academy for Engineering and Technology, Fudan University（复旦大学工程技术学院）；Jilin Provincial Key Laboratory of Intelligence Science and Engineering（吉林省智能科学与工程重点实验室）；Engineering Research Center of AI and Robotics, Ministry of Education（教育部人工智能与机器人工程研究中心）
National Engineering Laboratory for Big Data Analytics（大数据分析国家工程实验室）；Department of Computer Science, University of Massachusetts Boston（马萨诸塞大学波士顿分校计算机科学系）；Lenovo Research（联想研究院）；Department of Engineering, University of Massachusetts Boston（马萨诸塞大学波士顿分校工程系）
Generalized Category Discovery (GCD) aims to recognize both known and novel categories from a set of unlabeled data, based on another dataset labeled with only known categories. Without considering differences between known and novel categories, current methods learn about them in a coupled manner, which can hurt model's generalization and discriminative ability. Furthermore, the coupled training approach prevents these models transferring category-specific knowledge explicitly from labeled data to unlabeled data, which can lose high-level semantic information and impair model performance. To …
Given a single scene image, this paper proposes a method of Category-level 6D Object Pose and Size Estimation (COPSE) from the point cloud of the target object, without external real pose-annotated training data. Specifically, beyond the visual cues in RGB images, we rely on the shape information predominately from the depth (D) channel. The key idea is to explore the shape alignment of each instance against its corresponding category-level template shape, and the symmetric correspondence of each object category for estimating a coarse 3D object shape. Our framework deforms the point cloud of …
In this paper, we propose a novel 3D graph convolution based pipeline for category-level 6D pose and size estimation from monocular RGB-D images. The proposed method leverages an efficient 3D data augmentation and a novel vector-based decoupled rotation representation. Specifically, we first design an orientation-aware autoencoder with 3D graph convolution for latent feature learning. The learned latent feature is insensitive to point shift and size thanks to the shift and scale-invariance properties of the 3D graph convolution. Then, to efficiently decode the rotation information from the lat…
We propose a novel deep neural network architecture for semi-supervised semantic segmentation using heterogeneous annotations. Contrary to existing approaches posing semantic segmentation as a single task of region-based classification, our algorithm decouples classification and segmentation, and learns a separate network for each task. In this architecture, labels associated with an image are identified by classification network, and binary segmentation is subsequently performed for each identified label in segmentation network. The decoupled architecture enables us to learn classification an…
Incorporating encoding-decoding nets with adversarial nets has been widely adopted in image generation tasks. We observe that the state-of-the-art achievements were obtained by carefully balancing the reconstruction loss and adversarial loss, and such balance shifts with different network structures, datasets, and training strategies. Empirical studies have demonstrated that an inappropriate weight between the two losses may cause instability, and it is tricky to search for the optimal setting, especially when lacking prior knowledge on the data and network. This paper gives the first attempt …
Renmin University of China（中国人民大学）；Nanjing University of Science and Technology（南京理工大学）；Nreal；Tsinghua University（清华大学）
Recently, RGBD-based category-level 6D object pose estimation has achieved promising improvement in performance, however, the requirement of depth information prohibits broader applications. In order to relieve this problem, this paper proposes a novel approach named Object Level Depth reconstruction Network (OLD-Net) taking only RGB images as input for category-level 6D object pose estimation. We propose to directly predict object-level depth from a monocular RGB image by deforming the category-level shape prior into object-level depth and the canonical NOCS representation. Two novel modules …
South China University of Technology（华南理工大学）；Lenovo Research（联想研究院）
Text recognition has attracted considerable research interests because of its various applications. The cutting-edge text recognition methods are based on attention mechanisms. However, most of attention methods usually suffer from serious alignment problem due to its recurrency alignment operation, where the alignment relies on historical decoding results. To remedy this issue, we propose a decoupled attention network (DAN), which decouples the alignment operation from using historical decoding results. DAN is an effective, flexible and robust end-to-end text recognizer, which consists of thr…
Training neural networks with back-propagation (BP) requires a sequential passing of activations and gradients, which forces the network modules to work in a synchronous fashion. This has been recognized as the lockings (i.e., the forward, backward and update lockings) inherited from the BP. In this paper, we propose a fully decoupled training scheme using delayed gradients (FDG) to break all these lockings. The FDG splits a neural network into multiple modules and trains them independently and asynchronously using different workers (e.g., GPUs). We also introduce a gradient shrinking process …
Stanford University（斯坦福大学）；University of Science and Technology of China（中国科学技术大学）；Microsoft Research（微软研究院）；Wormpex AI Research（Wormpex人工智能研究院）；Peking University（北京大学）
Many different deep networks have been used to approximate, accelerate or improve traditional image operators. Among these traditional operators, many contain parameters which need to be tweaked to obtain the satisfactory results, which we refer to as parameterized image operators. However, most existing deep networks trained for these operators are only designed for one specific parameter configuration, which does not meet the needs of real scenarios that usually require flexible parameters settings. To overcome this limitation, we propose a new decoupled learning algorithm to learn from the …
University of Chinese Academy of Sciences（中国科学院大学）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；CAS Center for Excellence in Brain Science and Intelligence Technology（中国科学院脑科学与智能技术卓越创新中心）
Dynamic skeletal data, represented as the 2D/3D coordinates of human joints, has been widely studied for human action recognition due to its high-level semantic information and environmental robustness. However, previous methods heavily rely on designing hand-crafted traversal rules or graph topologies to draw dependencies between the joints, which are limited in performance and generalizability. In this work, we present a novel decoupled spatial-temporal attention network(DSTA-Net) for skeleton-based action recognition. It involves solely the attention blocks, allowing for modeling spatial-te…
Rethinking Unsupervised Cross-modal Flow Estimation: Learning from Decoupled Optimization and Consistency Constraint
College of Information Science and Electronic Engineering, Zhejiang University（浙江大学信息科学与电子工程学院）；Ningbo Global Innovation Center, Zhejiang University（浙江大学宁波全球创新中心）；NingboTech University（宁波科技大学）
Tsinghua University（清华大学）；Google（谷歌）；Technical University of Munich（慕尼黑工业大学）
While 6D object pose estimation has recently made a huge leap forward, most methods can still only handle a single or a handful of different objects, which limits their applications. To circumvent this problem, category-level object pose estimation has recently been revamped, which aims at predicting the 6D pose as well as the 3D metric size for previously unseen instances from a given set of object classes. This is, however, a much more challenging task due to severe intra-class shape variations. To address this issue, we propose GPV-Pose, a novel framework for robust category-level pose esti…
Cross-domain object detection is more challenging than object classification since multiple objects exist in an image and the location of each object is unknown in the unlabeled target domain. As a result, when we adapt features of different objects to enhance the transferability of the detector, the features of the foreground and the background are easy to be confused, which may hurt the discriminability of the detector. Besides, previous methods focused on category adaptation but ignored another important part for object detection, i.e., the adaptation on bounding box regression. To this end…
South China University of Technology（华南理工大学）；University of California at Merced（加州大学默塞德分校）；Google Research（谷歌研究院）；Huazhong University of Science and Technology（华中科技大学）；Yonsei University（延世大学）
Convolution is one of the basic building blocks of CNN architectures. Despite its common use, standard convolution has two main shortcomings: Content-agnostic and Computation-heavy. Dynamic filters are content-adaptive, while further increasing the computational overhead. Depth-wise convolution is a lightweight variant, but it usually leads to a drop in CNN performance or requires a larger number of channels. In this work, we propose the Decoupled Dynamic Filter (DDF) that can simultaneously tackle both of these shortcomings. Inspired by recent advances in attention, DDF decouples a depth-wise…
Georgia Institute of Technology（佐治亚理工学院）；NVIDIA（英伟达）；Emory University（埃默里大学）；Tsinghua University（清华大学）；Ant Financial（蚂蚁金融）
Inner product-based convolution has been a central component of convolutional neural networks (CNNs) and the key to learning visual representations. Inspired by the observation that CNN-learned features are naturally decoupled with the norm of features corresponding to the intra-class variation and the angle corresponding to the semantic difference, we propose a generic decoupled learning framework which models the intra-class variation and semantic difference independently. Specifically, we first reparametrize the inner product to a decoupled form and then generalize it to the decoupled convo…
Recently, density map regression-based methods have dominated in crowd counting owing to their excellent fitting ability on density distribution. However, further improvement tends to saturate mainly because of the confusing background noise and the large density variation. In this paper, we propose a Hierarchically Decoupled Network (HDNet) to solve the above two problems within a unified framework. Specifically, a background classification sub-task is decomposed from the density map prediction task, which is then assigned to a Density Decoupling Module (DDM) to exploit its highly discriminat…
University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；University of Technology Sydney（悉尼科技大学）；IBM Research（IBM研究院）；University of Oregon（俄勒冈大学）
In this paper, we analyze failure cases of state-of-the-art detectors and observe that most hard false positives result from classification instead of localization and they have a large negative impact on the performance of object detectors. We conjecture there are three factors: (1) Shared feature representation is not optimal due to the mismatched goals of feature learning for classification and localization; (2) multi-task learning helps, yet optimization of the multi-task loss may result in sub-optimal for individual tasks; (3) large receptive field for different scales leads to redundant …
School of Computer Science Faculty of Engineering University of Sydney（悉尼大学工程学院计算机科学学院）
Knowledge distillation is a widely used paradigm for inheriting information from a complicated teacher network to a compact student network and maintaining the strong performance. Different from image classification, object detectors are much more sophisticated with multiple loss functions in which features that semantic information rely on are tangled. In this paper, we point out that the information of features derived from regions excluding objects are also essential for distilling the student detector, which is usually ignored in existing approaches. In addition, we elucidate that features…
High-Resolution Underwater Camouflaged Object Detection: GBU-UCOD Dataset and Topology-Aware and Frequency-Decoupled Networks
College of Shipbuilding Engineering, Harbin Engineering University（哈尔滨工程大学船舶工程学院）；School of Information Science and Technology, Great Bay University（广东工业大学信息科学与技术学院）
Spatio-temporal Decoupled Knowledge Compensator for Few-Shot Action Recognition
School of Computer Science and Engineering, Nanjing University of Science and Technology（计算机科学与工程学院，南京理工大学）；State Key Lab of Brain-Machine Intelligence, Zhejiang University（脑机智能国家重点实验室，浙江大学）
FDNet: Feature Decoupled Segmentation Network for Tooth CBCT Image
Hangzhou Dianzi University（杭州电子科技大学）；College of Media Engineering, Communication University of Zhejiang（浙江传媒学院媒体工程学院）；Shandong University（山东大学）；University of Texas Southwestern Medical Center（德克萨斯大学西南医学中心）
针对牙齿 CBCT 中伪影复杂、边界模糊的问题，本文提出 FDNet，用 LF-Wavelet 增强全局语义并结合 SAM 编码器细化边界，在基准上取得最优分割性能。
Unsupervised domain adaptation in semantic segmentation has been raised to alleviate the reliance on expensive pixel-wise annotations. It leverages a labeled source domain dataset as well as unlabeled target domain images to learn a segmentation network. In this paper, we observe two main issues of the existing domain-invariant learning framework. (1) Being distracted by the feature distribution alignment, the network cannot focus on the segmentation task. (2) Fitting source domain data well would compromise the target domain performance. To address these issues, we propose DecoupleNet that al…
John Hopkins University（约翰霍普金斯大学）；City University of Hong Kong（香港城市大学）；UCLA（加州大学洛杉矶分校）
Many objects, especially these made by humans, are symmetric, e.g. cars and aeroplanes. This paper addresses the estimation of 3D structures of symmetric objects from multiple images of the same object category, e.g. different cars, seen from various viewpoints. We assume that the deformation between different instances from the same object category is non-rigid and symmetric. In this paper, we extend two leading non-rigid structure from motion (SfM) algorithms to exploit symmetry constraints. We model the both methods as energy minimization, in which we also recover the missing observations c…
School of Control and Computer Engineering, North China Electric Power University（华北电力大学控制与计算机工程学院）；China Mobile (Hangzhou) Information Technology Co., Ltd.（中国移动（杭州）信息技术有限公司）
As the scale of object detection dataset is smaller than that of image recognition dataset ImageNet, transfer learning has become a basic training method for deep learning object detection models, which will pretrain the backbone network of object detection model on ImageNet dataset to extract features for classification and localization subtasks. However, the classification task focuses on the salient region features of object, while the location task focuses on the edge features of object, so there is certain deviation between the features extracted by pretrained backbone network and the fea…
DPDETR: Decoupled Position Detection Transformer for Infrared-Visible Object Detection
School of Communications and Information Engineering, Chongqing University of Posts and Telecommunications（通信与信息工程学院，重庆邮电大学）；School of Intelligent Systems Engineering, Shenzhen Campus of Sun Yat-sen University（智能系统工程学院，中山大学深圳校区）；School of Mathematics and Statistics, Xi’an Jiaotong University（数学与统计学院，西安交通大学）
CAD-SLAM: Consistency-Aware Dynamic SLAM with Dynamic-Static Decoupled Mapping
Department of Automation, Shanghai Jiao Tong University（自动化系，上海交通大学）；Department of Computer Science, University College London（计算机科学系，伦敦大学学院）；Department of Engineering, University of Cambridge（工程系，剑桥大学）；Department of Automation, Key Laboratory of System Control and Information Processing of Ministry of Education, State Key Laboratory of Avionics Integration and Aviation System-of-Systems Synthesis, Shanghai Key Laboratory of Navigation and Location Based Services, Shanghai Jiao Tong University（自动化系，教育部系统控制与信息处理重点实验室，航空系统集成与航空系统-of-系统综合国家重点实验室，导航与基于位置的服务上海实验室，上海交通大学）
Boundary-aware Decoupled Flow Networks for Realistic Extreme Rescaling
Tsinghua Shenzhen International Graduate School, Tsinghua University（清华大学深圳国际研究生院）；College of Computer Science and Software Engineering, Shenzhen University（深圳大学计算机与软件学院）；WeChat Pay Lab33, Tencent（腾讯微信支付实验室）；Research Center of Artificial Intelligence, Peng Cheng Laboratory（鹏城实验室人工智能研究中心）
Hainan University（海南大学）；Wuhan University of Technology（武汉理工大学）；Beijing Jiaotong University（北京交通大学）；Shenzhen University（深圳大学）；City University of Hong Kong（香港城市大学）；City University of Hong Kong Shenzhen Research Institute（香港城市大学深圳研究院）
Stereo superpixel segmentation aims at grouping the discretizing pixels into perceptual regions through left and right views more collaboratively and efficiently. Existing superpixel segmentation algorithms mostly utilize color and spatial features as input, which may impose strong constraints on spatial information while utilizing the disparity information in terms of stereo image pairs. To alleviate this issue, we propose a stereo superpixel segmentation method with a decoupling mechanism of spatial information in this work. To decouple stereo disparity information and spatial information, t…
Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）；University of Chinese Academy of Sciences（中国科学院大学）；University of California, Los Angeles（加利福尼亚大学洛杉矶分校）；University of Southern California（南加利福尼亚大学）；Lenovo Research（联想研究院）；Lenovo Ltd.（联想有限公司）
Purpose: Automated liver tumor segmentation from Computed Tomography (CT) images is a necessary prerequisite in the interventions of hepatic abnormalities and surgery planning. However, accurate liver tumor segmentation remains challenging due to the large variability of tumor sizes and inhomogeneous texture. Recent advances based on Fully Convolutional Network (FCN) for medical image segmentation drew on the success of learning discriminative pyramid features. In this paper, we propose a Decoupled Pyramid Correlation Network (DPC-Net) that exploits attention mechanisms to fully leverage both …
Style-Decoupled Adaptive Routing Network for Underwater Image Enhancement
State Key Laboratory of Information Engineering in Surveying, Mapping and Remote Sensing, Wuhan University（测绘遥感信息工程国家重点实验室，武汉大学）；Department of Aeronautical and Aviation Engineering, The Hong Kong Polytechnic University（航空工程系，香港理工大学）
Decouple, Reorganize, and Fuse: A Multimodal Framework for Cancer Survival Prediction
College of Computer Science and Technology, Zhejiang University（浙江大学计算机科学与技术学院）；State Key Laboratory of Transvascular Implantation Devices of the Second Affiliated Hospital, Zhejiang University School of Medicine（浙江大学医学院第二附属医院血管植入设备国家重点实验室）；Transvascular Implantation Devices Research Institute（血管植入设备研究院）；Zhejiang Key Laboratory of Medical Imaging Artificial Intelligence（浙江省医学影像人工智能重点实验室）；China Mobile (Zhejiang) Research & Innovation Institute（中国移动（浙江）研究院）
Current CNN-based algorithms for recovering the 3D pose of an object in an image assume knowledge about both the object category and its 2D localization in the image. In this paper, we relax one of these constraints and propose to solve the task of joint object category and 3D pose estimation from an image assuming known 2D localization. We design a new architecture for this task composed of a feature network that is shared between subtasks, an object categorization network built on top of the feature network, and a collection of category dependent pose regression networks. We also introduce s…
Huazhong University of Science and Technology（华中科技大学）；University of Technology Sydney（悉尼科技大学）；Baidu Research（百度研究院）
The goal of person search is to localize and match query persons from scene images. For high efficiency, one-step methods have been developed to jointly handle the pedestrian detection and identification sub-tasks using a single network. There are two major challenges in the current one-step approaches. One is the mutual interference between the optimization objectives of multiple sub-tasks. The other is the sub-optimal identification feature learning caused by small batch size when end-to-end training. To overcome these problems, we propose a decoupled and memory-reinforced network (DMRNet). …
FD-DB: Frequency-Decoupled Dual-Branch Network for Unpaired Synthetic-to-Real Domain Translation
State Key Laboratory of Fluid Power and Mechatronic System, School of Mechanical Engineering, Zhejiang University（流体动力与机械系统国家重点实验室，机械工程学院，浙江大学）；Zhejiang Key Laboratory of Advanced Manufacturing Technology, School of Mechanical Engineering, Zhejiang University（浙江省先进制造技术重点实验室，机械工程学院，浙江大学）
View-decoupled Transformer for Person Re-identification under Aerial-ground Camera Network
School of Computer Science and Engineering, Sun Yat-Sen University（中山大学计算机科学与工程学院）；Department of Electrical and Computer Engineering, Johns Hopkins University（约翰霍普金斯大学电气与计算机工程系）；Pazhou Lab (HuangPu)（琶洲实验室（黄埔））；Guangdong Province Key Laboratory of Information Security Technology（广东省信息安全技术重点实验室）；Key Laboratory of Machine Intelligence and Advanced Computing, Ministry of Education（机器智能与先进计算教育部重点实验室）
University of Birmingham（伯明翰大学）；National University of Defense Technology（国防科技大学）
This article presents a semantic tracker which simultaneously tracks a single target and recognises its category. In general, it is hard to design a tracking model suitable for all object categories, e.g., a rigid tracker for a car is not suitable for a deformable gymnast. Category-based trackers usually achieve superior tracking performance for the objects of that specific category, but have difficulties being generalised. Therefore, we propose a novel unified robust tracking framework which explicitly encodes both generic features and category-based features. The tracker consists of a shared…
University of Technology Sydney（悉尼科技大学）；Beijing Jiaotong University（北京交通大学）；Zhibian Technology Co. Ltd.（之变科技有限公司）；Zhejiang University（浙江大学）
Superpixel segmentation has recently seen important progress benefiting from the advances in differentiable deep learning. However, the very high-resolution superpixel segmentation still remains challenging due to the expensive memory and computation cost, making the current advanced superpixel networks fail to process. In this paper, we devise Patch Calibration Networks (PCNet), aiming to efficiently and accurately implement high-resolution superpixel segmentation. PCNet follows the principle of producing high-resolution output from low-resolution input for saving GPU memory and relieving com…
