A key technical challenge in performing 6D object pose estimation from RGB-D image is to fully leverage the two complementary data sources. Prior works either extract information from the RGB image and depth separately or use costly post-processing steps, limiting their performances in highly cluttered scenes and real-time applications. In this work, we present DenseFusion, a generic framework for estimating 6D pose of a set of known objects from RGB-D images. DenseFusion is a heterogeneous architecture that processes the two data sources individually and uses a novel dense fusion network to extract pixel-wise dense feature embedding, from which the pose is estimated. Furthermore, we integrate an end-to-end iterative pose refinement procedure that further improves the pose estimation while achieving near real-time inference. Our experiments show that our method outperforms state-of-the-art approaches in two datasets, YCB-Video and LineMOD. We also deploy our proposed method to a real robot to grasp and manipulate objects based on the estimated pose.
IT-TextFusion: Iterative Text-Image Interaction with Text-Guided Residual Refinement for Degradation-Aware Image Fusion
School of Automation, Southeast University（东南大学自动化学院）；Chair of Robotics, Artificial Intelligence and Real-time Systems, Technical University of Munich（慕尼黑工业大学机器人、人工智能与实时系统研究所）
Shanghai Institute for Advanced Communication and Data Science, Shanghai University（上海大学上海先进通信与数据科学研究院）
Although recent works based on deep learning have made progress in improving recognition accuracy on scene text recognition, how to handle low-quality text images in end-to-end deep networks remains a research challenge. In this paper, we propose an Iterative Fusion based Recognizer (IFR) for low quality scene text recognition, taking advantage of refined text images input and robust feature representation. IFR contains two branches which focus on scene text recognition and low quality scene text image recovery respectively. We utilize an iterative collaboration between two branches, which can…
Apple（苹果公司）；Tsinghua University（清华大学）；University of Science and Technology Beijing（北京科技大学）
Video Frame Interpolation synthesizes non-existent images between adjacent frames, with the aim of providing a smooth and consistent visual experience. Two approaches for solving this challenging task are optical flow based and kernel-based methods. In existing works, optical flow based methods can provide accurate point-to-point motion description, however, they lack constraints on object structure. On the contrary, kernel-based methods focus on structural alignment, which relies on semantic and apparent features, but tends to blur results. Based on these observations, we propose a structure-…
School of Electrical and Information Engineering, Jiangsu University, Zhenjiang, 212013, China（江苏大学电气与信息工程学院）；School of Computer Science and Engineering, Jiangsu University of Science and Technology, Zhenjiang, 212003, China（江苏科技大学计算机科学与工程学院）；School of Automation, Southeast University, Nanjing, 210096, China（东南大学自动化学院）；Department of Computer Science and Engineering, University of North Texas, Denton, TX 76207, USA（北卡罗来纳州立大学计算机科学与工程系）；School of Computing, Nanjing Normal University, Nanjing, 210046, Jiangsu（南京师范大学计算机学院）
The search for specific objects or motifs is essential to art history as both assist in decoding the meaning of artworks. Digitization has produced large art collections, but manual methods prove to be insufficient to analyze them. In the following, we introduce an algorithm that allows users to search for image regions containing specific motifs or objects and find similar regions in an extensive dataset, helping art historians to analyze large digitized art collections. Computer vision has presented efficient methods for visual instance retrieval across photographs. However, applied to art c…
LIDIA: Precise Liver Tumor Diagnosis on Multi-Phase Contrast-Enhanced CT via Iterative Fusion and Asymmetric Contrastive Learning
College of Computer Science, Sichuan University（四川大学计算机学院）；DAMO Academy, Alibaba Group（阿里巴巴集团达摩院）；Hupan Lab（湖畔实验室）；Shengjing Hospital of China Medical University（中国医科大学附属盛京医院）；First Affiliated Hospital of Zhejiang University（浙江大学附属第一医院）
Monocular Depth Estimation and Segmentation for Transparent Object with Iterative Semantic and Geometric Fusion
School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）；Institute of Automation of Chinese Academy of Sciences（中国科学院自动化研究所）；School of Information Science and Technology, Shijiazhuang Tiedao University（石家庄铁道大学信息科学与技术学院）；D-Robotics（杭州宇树科技有限公司）
Bosch Center for Artificial Intelligence（博世人工智能中心）；University of Tübingen（图宾根大学）；Robert Bosch GmbH（罗伯特·博世有限公司）；Karlsruhe Institute of Technology（卡尔斯鲁厄理工学院）
Estimating 6D poses of objects is an essential computer vision task. However, most conventional approaches rely on camera data from a single perspective and therefore suffer from occlusions. We overcome this issue with our novel multi-view 6D pose estimation method called MV6D which accurately predicts the 6D poses of all objects in a cluttered scene based on RGB-D images from multiple perspectives. We base our approach on the PVN3D network that uses a single RGB-D image to predict keypoints of the target objects. We extend this approach by using a combined point cloud from multiple views and …
China University of Petroleum, Beijing Campus (CUP)（中国石油大学（北京））
The extensive research leveraging RGB-D information has been exploited in salient object detection. However, salient visual cues appear in various scales and resolutions of RGB images due to semantic gaps at different feature levels. Meanwhile, similar salient patterns are available in cross-modal depth images as well as multi-scale versions. Cross-modal fusion and multi-scale refinement are still an open problem in RGB-D salient object detection task. In this paper, we begin by introducing top-down and bottom-up iterative refinement architecture to leverage multi-scale features, and then devi…
Relative-Absolute Fusion: Rethinking Feature Extraction in Image-Based Iterative Method Selection for Solving Sparse Linear Systems
School of Software Technology, Dalian University of Technology（大连理工大学软件学院）；Greater Bay Area National Center of Technology Innovation（粤港澳大湾区国家技术创新中心）；CATARC Automotive Test Center (Tianjin) Co., Ltd（CATARC汽车测试中心（天津）有限公司）
RIKEN（理化学研究所）；RIKEN Center for Advanced Intelligence Project（理化学研究所先进智能项目中心）；University of Tokyo（东京大学）；Nokia Bell Labs（诺基亚贝尔实验室）
Coded aperture snapshot spectral imaging (CASSI) is a promising technique to capture the three-dimensional hyperspectral image (HSI) using a single coded two-dimensional (2D) measurement, in which algorithms are used to perform the inverse problem. Due to the ill-posed nature, various regularizers have been exploited to reconstruct the 3D data from the 2D measurement. Unfortunately, the accuracy and computational complexity are unsatisfied. One feasible solution is to utilize additional information such as the RGB measurement in CASSI. Considering the combined CASSI and RGB measurement, in thi…
The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））；Shenzhen Institute of Artificial Intelligence and Robotics for Society（深圳人工智能与机器人研究院）
3D semantic scene completion and 2D semantic segmentation are two tightly correlated tasks that are both essential for indoor scene understanding, because they predict the same semantic classes, using positively correlated high-level features. Current methods use 2D features extracted from early-fused RGB-D images for 2D segmentation to improve 3D scene completion. We argue that this sequential scheme does not ensure these two tasks fully benefit each other, and present an Iterative Mutual Enhancement Network (IMENet) to solve them jointly, which interactively refines the two tasks at the late…
This report presents the results of a multi-scale wavelet based scheme for single image de-hazing and underwater image enhancement. The scheme is fast and highly localized in addition to global enhancement of hazy images. A PDE-based formulation enables additional versatility as the iterative nature allows more flexibility for various types of images. Visual and objective results from experiments indicate that the proposed approach competes favourably or surpasses most of the state-of-the-art approaches.
FGS-SLAM: Fourier-based Gaussian Splatting for Real-time SLAM with Sparse and Dense Map Fusion
State Key Laboratory of Robotics, Shenyang Institute of Automation, Chinese Academy of Sciences（中国科学院沈阳自动化研究所机器人重点实验室）；University of Chinese Academy of Sciences（中国科学院大学）；Department of Mechanical and Energy Engineering, Southern University of Science and Technology（南方科技大学机械与能源工程系）
Nanjing University of Science and Technology（南京理工大学）
This paper proposes a Rotation-equivariant Attention Feature Fusion Pyramid Networks for Aerial Object Detection named ReAFFPN. ReAFFPN aims at improving the effect of rotation-equivariant features fusion between adjacent layers which suffers from the semantic and scale discontinuity. Due to the particularity of rotational equivariant convolution, general methods are unable to achieve their original effect while ensuring rotation equivariance of the network. To solve this problem, we design a new Rotation-equivariant Channel Attention which has the ability to both generate channel attention an…
Self-Supervised Diffusion MRI Denoising via Iterative and Stable Refinement
School of Biomedical Engineering, Division of Life Sciences and Medicine, USTC（中国科学技术大学生命科学与医学部生物医学工程学院）；MIRACLE Center, Suzhou Institute for Advance Research, USTC（中国科学技术大学苏州研究院MIRACLE中心）；State Key Laboratory of Precision and Intelligent Chemistry, USTC（中国科学技术大学精准与智能化学国家重点实验室）；Key Laboratory of Intelligent Information Processing of CAS, Institute of Computing Technology, CAS（中国科学院计算技术研究所中国科学院智能信息处理重点实验室）
Task-driven Image Fusion with Learnable Fusion Loss
Xi’an Jiaotong University（西安交通大学）；ETH Zürich（苏黎世联邦理工学院）；City University of Hong Kong（香港城市大学）；Tsinghua University（清华大学）；Northwestern Polytechnical University（西北工业大学）
Hefei Institute of Physical Science, Chinese Academy of Sciences（中国科学院合肥物质科学研究院）；University of Science and Technology of China（中国科学技术大学）；Nanyang Technological University（南洋理工大学）；Aerospace Information Research Institute, Chinese Academy of Sciences（中国科学院空天信息创新研究院）
Panchromatic (PAN) and multi-spectral (MS) image fusion, named Pan-sharpening, refers to super-resolve the low-resolution (LR) multi-spectral (MS) images in the spatial domain to generate the expected high-resolution (HR) MS images, conditioning on the corresponding high-resolution PAN images. In this paper, we present a simple yet effective \textit{alternating reverse filtering network} for pan-sharpening. Inspired by the classical reverse filtering that reverses images to the status before filtering, we formulate pan-sharpening as an alternately iterative reverse filtering process, which fus…
Wangxuan Institute of Computer Technology, Peking University（北京大学王选计算机研究所）
Unsupervised object discovery (UOD) has recently shown encouraging progress with the adoption of pre-trained Transformer features. However, current methods based on Transformers mainly focus on designing the localization head (e.g., seed selection-expansion and normalized cut) and overlook the importance of improving Transformer features. In this work, we handle UOD task from the perspective of feature enhancement and propose FOReground guidance and MUlti-LAyer feature fusion for unsupervised object discovery, dubbed FORMULA. Firstly, we present a foreground guidance strategy with an off-the-s…
Infrared and visible image fusion (IVIF) expects to obtain images that retain thermal radiation information from infrared images and texture details from visible images. In this paper, a model-based convolutional neural network (CNN) model, referred to as Algorithm Unrolling Image Fusion (AUIF), is proposed to overcome the shortcomings of traditional CNN-based IVIF models. The proposed AUIF model starts with the iterative formulas of two traditional optimization models, which are established to accomplish two-scale decomposition, i.e., separating low-frequency base information and high-frequen…
Mask-adaptive Gated Convolution and Bi-directional Progressive Fusion Network for Depth Completion
College of Information Science and Engineering, Northeastern University（东北大学信息科学与工程学院）；Foshan Graduate School of Innovation, Northeastern University（东北大学佛山创新研究生院）；National Frontiers Science Center for Industrial Intelligence and Systems Optimization, Northeastern University（东北大学工业智能与系统优化国家前沿科学中心）
FlowLUT: Efficient Image Enhancement via Differentiable LUTs and Iterative Flow Matching
School of Engineering, Zhejiang Normal University（工程学院，浙江师范大学）；School of Computer Science and Engineering, University of Science and Technology of China（计算机科学与工程学院，中国科学技术大学）；School of Computer Science and Engineering, Nanjing University of Science and Technology（计算机科学与工程学院，南京理工大学）；School of Information Science and Engineering, Shandong Normal University（信息科学与工程学院，山东师范大学）
提出 FlowLUT 端到端图像增强模型，结合多先验可微 3D LUT、动态融合与迭代流匹配，在保持 O(1) 高效色彩校正的同时恢复细节并消除伪影。
Bird-eye-view (BEV) based methods have made great progress recently in multi-view 3D detection task. Comparing with BEV based methods, sparse based methods lag behind in performance, but still have lots of non-negligible merits. To push sparse 3D detection further, in this work, we introduce a novel method, named Sparse4D, which does the iterative refinement of anchor boxes via sparsely sampling and fusing spatial-temporal features. (1) Sparse 4D Sampling: for each 3D anchor, we assign multiple 4D keypoints, which are then projected to multi-view/scale/timestamp image features to sample corres…
EPOFusion: Exposure aware Progressive Optimization Method for Infrared and Visible Image Fusion
College of Information Engineering, Zhejiang University of Technology（浙江工业大学信息工程学院）；College of Information Science and Technology, Zhejiang Shuren University（浙江树人大学信息科技学院）；Department of Electrical and Electronic Engineering, The University of Hong Kong（香港大学电机电子工程系）
Supervise-assisted Multi-modality Fusion Diffusion Model for PET Restoration
Beijing Institute of Technology（北京理工大学）；Research and Development Center of Agricultural Bank of China（中国农业银行研发中心）；Peking University（北京大学）；Beijing Normal University（北京师范大学）
Unaligned RGB Guided Hyperspectral Image Super-Resolution with Spatial-Spectral Concordance
Beijing Institute of Technology（北京理工大学）；Hangzhou Dianzi University（杭州电子科技大学）；Chinese Academy of Sciences（中国科学院）；University of Chinese Academy of Sciences（中国科学院大学）
AD-SRF: Attack-Defense Semantic Reinforcement for Multi-Modal Image Fusion
Electronic Information School, Wuhan University（武汉大学电子信息学院）；School of Robotics, Wuhan University（武汉大学机器人学院）；Suzhou Institute, Wuhan University（武汉大学苏州研究院）
