In this paper, we leverage image complexity as a prior for refining segmentation features to achieve accurate real-time semantic segmentation. The design philosophy is based on the observation that different pixel regions within an image exhibit varying levels of complexity, with higher complexities posing a greater challenge for accurate segmentation. We thus introduce image complexity as prior guidance and propose the Image Complexity prior-guided Feature Refinement Network (ICFRNet). This network aggregates both complexity and segmentation features to produce an attention map for refining segmentation features within an Image Complexity Guided Attention (ICGA) module. We optimize the network in terms of both segmentation and image complexity prediction tasks with a combined loss function. Experimental results on the Cityscapes and CamViD datasets have shown that our ICFRNet achieves higher accuracy with a competitive efficiency for real-time segmentation.
Recent deep learning based salient object detection methods which utilize both saliency and boundary features have achieved remarkable performance. However, most of them ignore the complementarity between saliency features and boundary features, thus get worse predictions in scenes with low contrast between foreground and background. To address this issue, we propose a novel Recurrent Two-Stream Guided Refinement Network (RTGRNet) that consists of iterating Two-Stream Guided Refinement Modules (TGRMs). TGRM consists of a Guide Block and two feature streams: saliency and boundary, the Guide Blo…
In this paper, we aim to develop an efficient and compact deep network for RGB-D salient object detection, where the depth image provides complementary information to boost performance in complex scenarios. Starting from a coarse initial prediction by a multi-scale residual block, we propose a progressively guided alternate refinement network to refine it. Instead of using ImageNet pre-trained backbone network, we first construct a lightweight depth stream by learning from scratch, which can extract complementary features more efficiently with less redundancy. Then, different from the existing…
Qualcomm AI Research（高通人工智能研究院）；Qualcomm Technologies, Inc.（高通技术公司）；Shanghai Jiao Tong University（上海交通大学）
There has been a recent surge of interest in introducing transformers to 3D human pose estimation (HPE) due to their powerful capabilities in modeling long-term dependencies. However, existing transformer-based methods treat body joints as equally important inputs and ignore the prior knowledge of human skeleton topology in the self-attention mechanism. To tackle this issue, in this paper, we propose a Pose-Oriented Transformer (POT) with uncertainty guided refinement for 3D HPE. Specifically, we first develop novel pose-oriented self-attention mechanism and distance-related position embedding…
Rutgers University（罗格斯大学）；Tencent Hippocrates Research Labs（腾讯希波克拉底研究实验室）；Lieber Institute for Brain Development（利伯大脑发育研究所）
Instance segmentation is of great importance for many biological applications, such as study of neural cell interactions, plant phenotyping, and quantitatively measuring how cells react to drug treatment. In this paper, we propose a novel box-based instance segmentation method. Box-based instance segmentation methods capture objects via bounding boxes and then perform individual segmentation within each bounding box region. However, existing methods can hardly differentiate the target from its neighboring objects within the same bounding box region due to their similar textures and low-contras…
University of Science and Technology Beijing（北京科技大学）；University of Central Florida（中佛罗里达大学）；Scoop Medical（Scoop医疗）
The encoder-decoder architecture is widely used as a lightweight semantic segmentation network. However, it struggles with a limited performance compared to a well-designed Dilated-FCN model for two major problems. First, commonly used upsampling methods in the decoder such as interpolation and deconvolution suffer from a local receptive field, unable to encode global contexts. Second, low-level features may bring noises to the network decoder through skip connections for the inadequacy of semantic concepts in early encoder layers. To tackle these challenges, a Global Enhancement Method is pro…
Peking University（北京大学）；Deepwise AI Lab（深睿医疗人工智能实验室）；Microsoft Research Asia（微软亚洲研究院）；The University of Hong Kong（香港大学）
As an important upstream task for many medical applications, supervised landmark localization still requires non-negligible annotation costs to achieve desirable performance. Besides, due to cumbersome collection procedures, the limited size of medical landmark datasets impacts the effectiveness of large-scale self-supervised pre-training methods. To address these challenges, we propose a two-stage framework for one-shot medical landmark localization, which first infers landmarks by unsupervised registration from the labeled exemplar to unlabeled targets, and then utilizes these noisy pseudo l…
Recently, most of the state-of-the-art human pose estimation methods are based on heatmap regression. The final coordinates of keypoints are obtained by decoding heatmap directly. In this paper, we aim to find a better approach to get more accurate localization results. We mainly put forward two suggestions for improvement: 1) different features and methods should be applied for rough and accurate localization, 2) relationship between keypoints should be considered. Specifically, we propose a two-stage graph-based and model-agnostic framework, called Graph-PCNN, with a localization subnet and …
Huawei Noah’s Ark Lab（华为诺亚方舟实验室）；South China University of Technology（华南理工大学）
Object detectors trained on fully-annotated data currently yield state of the art performance but require expensive manual annotations. On the other hand, weakly-supervised detectors have much lower performance and cannot be used reliably in a realistic setting. In this paper, we study the hybrid-supervised object detection problem, aiming to train a high quality detector with only a limited amount of fullyannotated data and fully exploiting cheap data with imagelevel labels. State of the art methods typically propose an iterative approach, alternating between generating pseudo-labels and upda…
RUFNet: Query-Guided Support Mask Refinement and Uncertainty Fusion based on Hybrid Mamba for Few-Shot Brain Tumor Segmentation
Department of Language Science and Technology, The Hong Kong Polytechnic University（香港理工大学语言科学与技术系）；School of Artificial Intelligence, Chongqing University of Technology（重庆理工大学人工智能学院）；Affiliated Lianyungang Hospital of Xuzhou Medical University（徐州医科大学附属连云港医院）；College of Mechatronics and Control Engineering, Shenzhen University（深圳大学机电与控制工程学院）
VLM-Guided Iterative Refinement for Surgical Image Segmentation with Foundation Models
School of Automation Science and Engineering, South China University of Technology, Guangzhou, China（自动化科学与工程学院，华南理工大学，广州，中国）；Vanderbilt University, Nashville, TN, USA（范德比尔特大学，纳什维尔，田纳西州，美国）；The Pennsylvania State University, University Park, PA , USA（宾夕法尼亚州立大学，大学园，宾夕法尼亚州，美国）；Peking University, Beijing , China（北京大学，北京，中国）；Virginia Commonwealth University, Richmond, VA, USA（弗吉尼亚共同wealth大学，里士满，弗吉尼亚州，美国）；University of California San Diego, San Diego, CA, USA（加州大学圣地亚哥分校，圣地亚哥，加利福尼亚州，美国）
3D Weakly Supervised Semantic Segmentation via Class-Aware and Geometry-Guided Pseudo-Label Refinement
College of Computer Science, Beihang University（北京航空航天大学计算机科学学院）；College of Computer Science and Software Engineering, Shenzhen University（深圳大学计算机科学与软件工程学院）；Department of Information Engineering and Computer Science, University of Trento（特伦托大学信息工程与计算机科学系）；Meituan（美团）
IT-TextFusion: Iterative Text-Image Interaction with Text-Guided Residual Refinement for Degradation-Aware Image Fusion
School of Automation, Southeast University（东南大学自动化学院）；Chair of Robotics, Artificial Intelligence and Real-time Systems, Technical University of Munich（慕尼黑工业大学机器人、人工智能与实时系统研究所）
RIGS-Refiner: Risk-Guided Recursive Refinement in Prediction Space for Colonoscopy Polyp Segmentation
Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究院）；South China University of Technology（华南理工大学）；CyPhi( Ψ Φ \Psi\Phi ) AI Research Lab, School of IT, Monash University Malaysia（马来西亚莫纳什大学信息技术学院CyPhi( Ψ Φ \Psi\Phi )人工智能研究实验室）
Blind face inpainting refers to the task of reconstructing visual contents without explicitly indicating the corrupted regions in a face image. Inherently, this task faces two challenges: (1) how to detect various mask patterns of different shapes and contents; (2) how to restore visually plausible and pleasing contents in the masked regions. In this paper, we propose a novel two-stage blind face inpainting method named Frequency-guided Transformer and Top-Down Refinement Network (FT-TDR) to tackle these challenges. Specifically, we first use a transformer-based network to detect the corrupted…
Anatomy-preserving unpaired cone-beam CT refinement for image-guided radiotherapy using pseudo-label guided diffusion
Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究院）；Southern University of Science and Technology（南方科技大学）；Huashan Hospital, Fudan University（复旦大学附属华山医院）
TSAR-MVS: Textureless-aware Segmentation and Correlative Refinement Guided Multi-View Stereo
Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）；Agricultural Information Institute, Chinese Academy of Agricultural Sciences（中国农业科学院农业信息研究所）
3D Hand Mesh-Guided AI-Generated Malformed Hand Refinement with Hand Pose Transformation via Diffusion Model
Department of Computer and Information Science, University of Macau, Macau, China（计算机与信息科学系，澳门大学，澳门，中国）；ShanghaiTech University, Shanghai, China（上海科技大学，上海，中国）
MGTraj: Multi-Granularity Goal-Guided Human Trajectory Prediction with Recursive Refinement Network
Division of Emerging Interdisciplinary Areas, The Hong Kong University of Science and Technology（新兴交叉领域分校，香港科学与技术大学）；Robotics and Autonomous Systems Thrust, The Hong Kong University of Science and Technology (Guangzhou)（机器人与自主系统方向，香港科学与技术大学（广州））
The Johns Hopkins University（约翰霍普金斯大学）；Adobe（Adobe公司）
We propose Mask Guided (MG) Matting, a robust matting framework that takes a general coarse mask as guidance. MG Matting leverages a network (PRN) design which encourages the matting model to provide self-guidance to progressively refine the uncertain regions through the decoding process. A series of guidance mask perturbation operations are also introduced in the training to further enhance its robustness to external guidance. We show that PRN can generalize to unseen types of guidance masks such as trimap and low-quality alpha matte, making it suitable for various application pipelines. In a…
Xidian University（西安电子科技大学）；Zhejiang Lab（之江实验室）；Chongqing University of Posts and Telecommunications（重庆邮电大学）
Unsupervised person re-identification (ReID) aims at learning discriminative identity features for person retrieval without any annotations. Recent advances accomplish this task by leveraging clustering-based pseudo labels, but these pseudo labels are inevitably noisy which deteriorate model performance. In this paper, we propose a Neighbour Consistency guided Pseudo Label Refinement (NCPLR) framework, which can be regarded as a transductive form of label propagation under the assumption that the prediction of each example should be similar to its nearest neighbours'. Specifically, the refined…
PhysVR: Vision-Language Model Guided Interference-aware Temporal Feature Refinement for Remote Physiological Measurement
School of Computer Science and Engineering, Nanjing University of Science and Technology（南京理工大学计算机科学与工程学院）；College of Computer Science and Technology, Qingdao University（青岛大学计算机科学技术学院）
Class-Aware Mask-Guided Feature Refinement for Scene Text Recognition
School of Electronic Information and Communications, Huazhong University of Science and Technology（华中科技大学电子信息与通信学院）；School of Artificial Intelligence and Automation, Huazhong University of Science and Technology（华中科技大学人工智能与自动化学院）；Huawei Inc.（华为公司）；School of Software Engineering, Huazhong University of Science and Technology（华中科技大学软件学院）
This paper presents StereoNet, the first end-to-end deep architecture for real-time stereo matching that runs at 60 fps on an NVidia Titan X, producing high-quality, edge-preserved, quantization-free disparity maps. A key insight of this paper is that the network achieves a sub-pixel matching precision than is a magnitude higher than those of traditional stereo matching approaches. This allows us to achieve real-time performance by using a very low resolution cost volume that encodes all the information needed to achieve high disparity precision. Spatial precision is achieved by employing a le…
CoDiR: Confidence-Guided Diffusion Refinement for Semi-Supervised Histopathology Segmentation
AI VIETNAM Lab（AI VIETNAM实验室）；Portland Community College（波特兰社区学院）；University of Science, VNU-HCM（胡志明市国家大学科学大学）；Hanoi University of Science and Technology（河内科技大学）；Jeonbuk National University（全北国立大学）；Washington University School of Medicine（华盛顿大学医学院）；Perelman School of Medicine, University of Pennsylvania（宾夕法尼亚大学佩雷尔曼医学院）
Monocular Depth Guided Occlusion-Aware Disparity Refinement via Semi-supervised Learning in Laparoscopic Images
School of Life Science and Technology, Harbin Institute of Technology（生命科学与技术学院，哈尔滨工业大学）；State Key Laboratory of Robotics and System, Harbin Institute of Technology（机器人与系统国家重点实验室，哈尔滨工业大学）
TU Wien（维也纳工业大学）；Automation and Control Institute（自动化与控制研究所）
Accurate and robust object pose estimation for robotics applications requires verification and refinement steps. In this work, we propose to integrate hypotheses verification with object pose refinement guided by physics simulation. This allows the physical plausibility of individual object pose estimates and the stability of the estimated scene to be considered in a unified optimization. The proposed method is able to adapt to scenes of multiple objects and efficiently focuses on refining the most promising object poses in multi-hypotheses scenarios. We call this integrated approach VeREFINE …
University of Electronic Science and Technology of China（电子科技大学）；King’s College London（伦敦国王学院）；University Hospitals KU Leuven（鲁汶大学附属医院）；University College London（伦敦大学学院）；SenseTime Research（商汤科技研究院）
Segmentation of the fetal brain from stacks of motion-corrupted fetal MRI slices is important for motion correction and high-resolution volume reconstruction. Although Convolutional Neural Networks (CNNs) have been widely used for automatic segmentation of the fetal brain, their results may still benefit from interactive refinement for challenging slices. To improve the efficiency of interactive refinement process, we propose an Uncertainty-Guided Interactive Refinement (UGIR) framework. We first propose a grouped convolution-based CNN to obtain multiple automatic segmentation predictions with…
