3D object reconstruction from a single-view image is a long-standing challenging problem. Previous work was difficult to accurately reconstruct 3D shapes with a complex topology which has rich details at the edges and corners. Moreover, previous works used synthetic data to train their network, but domain adaptation problems occurred when tested on real data. In this paper, we propose a Dynamic Multi-branch Information Fusion Network (DmifNet) which can recover a high-fidelity 3D shape of arbitrary topology from a 2D image. Specifically, we design several side branches from the intermediate layers to make the network produce more diverse representations to improve the generalization ability of network. In addition, we utilize DoG (Difference of Gaussians) to extract edge geometry and corners information from input images. Then, we use a separate side branch network to process the extracted data to better capture edge geometry and corners feature information. Finally, we dynamically fuse the information of all branches to gain final predicted probability. Extensive qualitative and quantitative experiments on a large-scale publicly available dataset demonstrate the validity and efficiency of our method. Code and models are publicly available at https://github.com/leilimaster/DmifNet.
REASON: Probability map-guided dual-branch fusion framework for gastric content assessment
School of Computer Science and Engineering, Hunan University of Science and Technology（计算机科学与工程学院，湖南科技大学）；State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences（多模态人工智能系统国家重点实验室，中国科学院自动化研究所）；School of Artificial Intelligence, University of Chinese Academy of Sciences（人工智能学院，中国科学院大学）；Department of Anesthesiology, Huashan Hospital, Fudan University（麻醉科，复旦大学附属华山医院）
The University of Queensland（昆士兰大学）；The Commonwealth Scientific and Industrial Research Organisation（联邦科学与工业研究组织）；University of Science and Technology of China（中国科学技术大学）；Chinese Academy of Sciences（中国科学院）；Institute of Computing Technology, CAS（中国科学院计算技术研究所）
Deep neural networks have been widely studied for predicting a medical condition, such as total knee replacement (TKR). It has shown that data of different modalities, such as imaging data, clinical variables and demographic information, provide complementary information and thus can improve the prediction accuracy together. However, the data sources of various modalities may not always be of high quality, and each modality may have only partial information of medical condition. Thus, predictions from different modalities can be opposite, and the final prediction may fail in the presence of su…
The University of Texas at Austin（德克萨斯大学奥斯汀分校）；Texas A&M University（德克萨斯农工大学）；Horizon Robotics Inc（地平线机器人公司）
We present AutoPose, a novel neural architecture search(NAS) framework that is capable of automatically discovering multiple parallel branches of cross-scale connections towards accurate and high-resolution 2D human pose estimation. Recently, high-performance hand-crafted convolutional networks for pose estimation show growing demands on multi-scale fusion and high-resolution representations. However, current NAS works exhibit limited flexibility on scale searching, they dominantly adopt simplified search spaces of single-branch architectures. Such simplification limits the fusion of informati…
Institute of Artificial Intelligence and Robotics, Xi’an Jiaotong University（西安交通大学人工智能与机器人研究所）；the Second Academy of CASIC（中国航天科工集团第二研究院）
Point clouds and images could provide complementary information when representing 3D objects. Fusing the two kinds of data usually helps to improve the detection results. However, it is challenging to fuse the two data modalities, due to their different characteristics and the interference from the non-interest areas. To solve this problem, we propose a Multi-Branch Deep Fusion Network (MBDF-Net) for 3D object detection. The proposed detector has two stages. In the first stage, our multi-branch feature extraction network utilizes Adaptive Attention Fusion (AAF) modules to produce cross-modal f…
Suzhou University of Science and Technology（苏州科技大学）；Jiangnan University（江南大学）
Background: Leaning redundant and complementary relationships is a critical step in the human visual system. Inspired by the infrared cognition ability of crotalinae animals, we design a joint convolution auto-encoder (JCAE) network for infrared and visible image fusion. Methods: Our key insight is to feed infrared and visible pair images into the network simultaneously and separate an encoder stream into two private branches and one common branch, the private branch works for complementary features learning and the common branch does for redundant features learning. We also build two fusion r…
Electronic Information School, Wuhan University（武汉大学电子信息学院）；School of Cyber Science and Engineering, Wuhan University（武汉大学网络空间安全学院）
Anti-counterfeiting QR codes are widely used in people's work and life, especially in product packaging. However, the anti-counterfeiting QR code has the risk of being copied and forged in the circulation process. In reality, copying is usually based on genuine anti-counterfeiting QR codes, but the brands and models of copiers are diverse, and it is extremely difficult to determine which individual copier the forged anti-counterfeiting code come from. In response to the above problems, this paper proposes a method for copy forgery identification of anti-counterfeiting QR code based on deep lea…
Deep learning is a rapidly developing approach in the field of infrared and visible image fusion. In this context, the use of dense blocks in deep networks significantly improves the utilization of shallow information, and the combination of the Generative Adversarial Network (GAN) also improves the fusion performance of two source images. We propose a new method based on dense blocks and GANs , and we directly insert the input image-visible light image in each layer of the entire network. We use SSIM and gradient loss functions that are more consistent with perception instead of mean square e…
Efficient Event Stream Super-Resolution with Recursive Multi-Branch Fusion
School of Computer Science and Engineering, Sun Yat-Sen University（中山大学计算机科学与工程学院）；Peng Cheng Laboratory（鹏城实验室）；Peking University（北京大学）；Shenzhen International Graduate School, Tsinghua University（清华大学深圳国际研究生院）
Wuhan University（武汉大学）；Tencent AI Lab（腾讯AI实验室）；University at Buffalo（纽约州立大学布法罗分校）
Accurate 3D reconstruction of the hand and object shape from a hand-object image is important for understanding human-object interaction as well as human daily activities. Different from bare hand pose estimation, hand-object interaction poses a strong constraint on both the hand and its manipulated object, which suggests that hand configuration may be crucial contextual information for the object, and vice versa. However, current approaches address this task by training a two-branch network to reconstruct the hand and object separately with little communication between the two branches. In th…
College of Safety Science and Engineering, Civil Aviation University of China（中国民航大学安全科学与工程学院）；College of Computer Science and Technology, Civil Aviation University of China（中国民航大学计算机科学与技术学院）；North China Electric Power University（华北电力大学）
CDDFuse: Correlation-Driven Dual-Branch Feature Decomposition for Multi-Modality Image Fusion
Xi’an Jiaotong University（西安交通大学）；ETH Zürich（苏黎世联邦理工学院）；Northwestern Polytechnical University（西北工业大学）；Research and Development Institute of Northwestern Polytechnical University in Shenzhen（西北工业大学深圳研究院）；Harvard University（哈佛大学）；University of Würzburg（维尔茨堡大学）
A Cross Branch Fusion-Based Contrastive Learning Framework for Point Cloud Self-supervised Learning
Karlsruhe Institute of Technology（卡尔斯鲁厄理工学院）；Tsinghua University（清华大学）；Fraunhofer Institute of Optronics, System Technologies and Image Exploitation IOSB（弗劳恩霍夫光电、系统技术和图像处理研究所）
Rethinking Early-Fusion Strategies for Improved Multispectral Object Detection
College of Information Science and Electronic Engineering, Zhejiang University（浙江大学信息科学与电子工程学院）；School of Information and Electrical Engineering, Hangzhou City University（杭州城市大学信息与电气工程学院）；Hangzhou City University Binjiang Innovation Center（杭州城市大学滨江创新中心）；Jinhua Institute of Zhejiang University（浙江大学金华研究院）
Shanghai Institute for Advanced Communication and Data Science, Shanghai University（上海大学上海先进通信与数据科学研究院）
Although recent works based on deep learning have made progress in improving recognition accuracy on scene text recognition, how to handle low-quality text images in end-to-end deep networks remains a research challenge. In this paper, we propose an Iterative Fusion based Recognizer (IFR) for low quality scene text recognition, taking advantage of refined text images input and robust feature representation. IFR contains two branches which focus on scene text recognition and low quality scene text image recovery respectively. We utilize an iterative collaboration between two branches, which can…
Multi-Branch Non-Homogeneous Image Dehazing via Concentration Partitioning and Image Fusion
School of Software and Internet of Things Engineering, Zhejiang Gongshang University（软件与物联网工程学院，浙江工商大学）；School of Computer Science and Technology, Tiangong University（计算机科学与技术学院，天工大学）
We present a novel lightweight convolutional neural network for point cloud analysis. In contrast to many current CNNs which increase receptive field by downsampling point cloud, our method directly operates on the entire point sets without sampling and achieves good performances efficiently. Our network consists of point voxel convolution (PVC) layer as building block. Each layer has two parallel branches, namely the voxel branch and the point branch. For the voxel branch specifically, we aggregate local features on non-empty voxel centers to reduce geometric information loss caused by voxeli…
MagicFuse: Single Image Fusion for Visual and Semantic Reinforcement
Electronic Information School, Wuhan University, China（武汉大学电子信息学院）；Suzhou Institute of Wuhan University, China（武汉大学苏州研究院）；School of Automation, Wuhan University, China（武汉大学自动化学院）
SenseTime Group Limited（商汤科技集团有限公司）；The Chinese University of HongKong（香港中文大学）
Although person re-identification (ReID) has achieved significant improvement recently by enforcing part alignment, it is still a challenging task when it comes to distinguishing visually similar identities or identifying the occluded person. In these scenarios, magnifying details in each part features and selectively fusing them together may provide a feasible solution. In this work, we propose MagnifierNet, a triple-branch network which accurately mines details from whole to parts. Firstly, the holistic salient features are encoded by a global branch. Secondly, to enhance detailed representa…
DAF-Net: A Dual-Branch Feature Decomposition Fusion Network with Domain Adaptive for Infrared and Visible Image Fusion
University of Electronic Science and Technology of China（电子科技大学）；School of Information and Communication Engineering（信息与通信工程学院）；School of Resources and Environment（资源与环境学院）
CMFN: Cross-Modal Fusion Network for Irregular Scene Text Recognition
Intelligent Software Research Center, Institute of Software Chinese Academy of Sciences（中国科学院软件研究所智能软件研究中心）；University of Chinese Academy of Sciences（中国科学院大学）；The Laboratory of Cognition and Decision Intelligence for Complex Systems, Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所复杂系统认知与决策智能实验室）；State Key Laboratory of Computer Science, Institute of Software Chinese Academy of Sciences（中国科学院软件研究所计算机科学国家重点实验室）
Despite recent advances in multi-scale deep representations, their limitations are attributed to expensive parameters and weak fusion modules. Hence, we propose an efficient approach to fuse multi-scale deep representations, called convolutional fusion networks (CFN). Owing to using 1$\times$1 convolution and global average pooling, CFN can efficiently generate the side branches while adding few parameters. In addition, we present a locally-connected fusion module, which can learn adaptive weights for the side branches and form a discriminatively fused feature. CFN models trained on the CIFAR …
Spatial-Frequency-Aware Implicit Neural Representation of Multidimensional Signals via MLP-KAN Fusion
Institute of Computational Imaging, Beijing Information Science and Technology University（北京信息科技大学计算成像研究所）；College of Computer Science (College of Software), Inner Mongolia University（内蒙古大学计算机学院（软件学院））；Research Center for Spatiotemporal Intelligence, Inner Mongolia University（内蒙古大学时空智能研究中心）
University of Kansas（堪萨斯大学）；Ryerson University（瑞尔森大学）
The paper focuses on improving the recent plug-and-play patch rescaling module (PRM) based approaches for crowd counting. In order to make full use of the PRM potential and obtain more reliable and accurate results for challenging images with crowd-variation, large perspective, extreme occlusions, and cluttered background regions, we propose a new PRM based multi-resolution and multi-task crowd counting network by exploiting the PRM module with more effectiveness and potency. The proposed model consists of three deep-layered branches with each branch generating feature maps of different resolu…
Beijing University of Posts and Telecommunications（北京邮电大学）；Academy of Broadcasting Science（广播科学研究院）
Convolutional neural networks (CNNs) have demonstrated superior performance in super-resolution (SR). However, most CNN-based SR methods neglect the different importance among feature channels or fail to take full advantage of the hierarchical features. To address these issues, this paper presents a novel recursive unit. Firstly, at the beginning of each unit, we adopt a compact channel attention mechanism to adaptively recalibrate the channel importance of input features. Then, the multi-level features, rather than only deep-level features, are extracted and fused. Additionally, we find that …
Shenzhen Institute of Artificial Intelligence and Robotics for Society（深圳人工智能与机器人研究院）；School of Science and Engineering, The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳）理工学院）
Recently, RGB-Thermal based perception has shown significant advances. Thermal information provides useful clues when visual cameras suffer from poor lighting conditions, such as low light and fog. However, how to effectively fuse RGB images and thermal data remains an open challenge. Previous works involve naive fusion strategies such as merging them at the input, concatenating multi-modality features inside models, or applying attention to each data modality. These fusion strategies are straightforward yet insufficient. In this paper, we propose a novel fusion method named Explicit Attention…
DualDiff: Dual-branch Diffusion Model for Autonomous Driving with Semantic Fusion
National Key Laboratory of Human-Machine Hybrid Augmented Intelligence（国家级人机混合增强智能实验室）；National Engineering Research Center for Visual Information and Applications（国家视觉信息与应用工程研究中心）；Institute of Artificial Intelligence and Robotics（人工智能与机器人研究所）；Xi’an Jiaotong University（西安交通大学）；University of Science and Technology of China（中国科学技术大学）
RXFOOD: Plug-in RGB-X Fusion for Object of Interest Detection
Cleveland State University（克利夫兰州立大学）；Centre for Frontier AI Research (CFAR)（前沿人工智能研究中心）；Agency for Science, Technology and Research (A*STAR)（新加坡科技研究局）；Institute of High Performance Computing (IHPC)（高性能计算研究所）；Brookhaven National Laboratory（布鲁克海文国家实验室）
DBCF: Dual-Branch Complementary Fusion of Foundation Models for Generalized Deepfake Detection
University of Chinese Academy of Sciences（中国科学院大学）；Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）；Ocean University of China（中国海洋大学）
Shanghai Jiao Tong University（上海交通大学）；University of Electronic Science and Technology of China（电子科技大学）；University of Calgary（卡尔加里大学）
The fusion of images taken by heterogeneous sensors helps to enrich the information and improve the quality of imaging. In this article, we present a hybrid model consisting of a convolutional encoder and a Transformer-based decoder to fuse multimodal images. In the encoder, a non-local cross-modal attention block is proposed to capture both local and global dependencies of multiple source images. A branch fusion module is designed to adaptively fuse the features of the two branches. We embed a Transformer module with linear complexity in the decoder to enhance the reconstruction capability of…
SARFusion: Scene-Aware Routing Fusion for Robust Camera-LiDAR 3D Object Detection
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）；Wuhan College（武汉学院）
College of Information Science and Engineering, Xinjiang University（新疆大学信息科学与工程学院）
Image super-resolution reconstruction achieves better results than traditional methods with the help of the powerful nonlinear representation ability of convolution neural network. However, some existing algorithms also have some problems, such as insufficient utilization of phased features, ignoring the importance of early phased feature fusion to improve network performance, and the inability of the network to pay more attention to high-frequency information in the reconstruction process. To solve these problems, we propose a multi-branch feature multiplexing fusion network with mixed multi-…
Cross-Dataset Experimental Study of Radar-Camera Fusion in Bird's-Eye View
Stellantis, Opel Automobile GmbH（斯泰兰蒂斯集团，欧宝汽车有限公司）；Rheinland-Pfälzische Technische Universität Kaiserslautern-Landau（莱茵兰-普法尔茨凯泽斯劳滕-兰道工业大学）；German Research Center for Artificial Intelligence（德国人工智能研究中心）
Jiangnan University（江南大学）；University of Surrey（萨里大学）
Deep learning-based image fusion approaches have obtained wide attention in recent years, achieving promising performance in terms of visual perception. However, the fusion module in the current deep learning-based methods suffers from two limitations, \textit{i.e.}, manually designed fusion function, and input-independent network learning. In this paper, we propose an unsupervised adaptive image fusion method to address the above issues. We propose a feature mutual mapping fusion module and dual-branch multi-scale autoencoder. More specifically, we construct a global map to measure the connec…
National Chung Hsing University（中兴大学）；National Taichung University of Science and Technology（台中科技大学）
Image restoration is a challenging and ill-posed problem which also has been a long-standing issue. However, most of learning based restoration methods are proposed to target one degradation type which means they are lack of generalization. In this paper, we proposed a multi-branch restoration model inspired from the Human Visual System (i.e., Retinal Ganglion Cells) which can achieve multiple restoration tasks in a general framework. The experiments show that the proposed multi-branch architecture, called CMFNet, has competitive performance results on four datasets, including image dehazing, …
Road detection based on remote sensing images is of great significance to intelligent traffic management. The performances of the mainstream road detection methods are mainly determined by their extracted features, whose richness and robustness can be enhanced by fusing features of different types and cross-layer connections. However, the features in the existing mainstream model frameworks are often similar in the same layer by the single-task training, and the traditional cross-layer fusion ways are too simple to obtain an efficient effect, so more complex fusion ways besides concatenation a…
Jiangnan University（江南大学）；The Hong Kong Polytechnic University（香港理工大学）
Traditional segmentation methods for colonic polyps are mainly designed based on low-level features. They could not accurately extract the location of small colonic polyps. Although the existing deep learning methods can improve the segmentation accuracy, their effects are still unsatisfied. To meet the above challenges, we propose a hybrid network called Fusion-Transformer-HardNetMSEG (i.e., Fu-TransHNet) in this study. Fu-TransHNet uses deep learning of different mechanisms to fuse each other and is enhanced with multi-view collaborative learning techniques. Firstly, the Fu-TransHNet utilize…
