Voxel-based 3D object classification has been thoroughly studied in recent years. Most previous methods convert the classic 2D convolution into a 3D form that will be further applied to objects with binary voxel representation for classification. However, the binary voxel representation is not very effective for 3D convolution in many cases. In this paper, we propose a hybrid cascade architecture for voxel-based 3D object classification. It consists of three stages composed of fully connected and convolutional layers, dealing with easy, moderate, and hard 3D models respectively. Both accuracy and speed can be balanced in our proposed method. By giving each voxel a signed distance value, an obvious gain regarding the accuracy can be observed. Besides, the mean inference time can be speeded up hugely compared with the state-of-the-art point cloud and voxel based methods.
Huazhong University of Science and Technology（华中科技大学）
Semantic scene completion (SSC) aims to complete a partial 3D scene and predict its semantics simultaneously. Most existing works adopt the voxel representations, thus suffering from the growth of memory and computation cost as the voxel resolution increases. Though a few works attempt to solve SSC from the perspective of 3D point clouds, they have not fully exploited the correlation and complementarity between the two tasks of scene completion and semantic segmentation. In our work, we present CasFusionNet, a novel cascaded network for point cloud semantic scene completion by dense feature fu…
This paper presents a learning-based, lossless compression method for static point cloud geometry, based on context-adaptive arithmetic coding. Unlike most existing methods working in the octree domain, our encoder operates in a hybrid mode, mixing octree and voxel-based coding. We adaptively partition the point cloud into multi-resolution voxel blocks according to the point cloud structure, and use octree to signal the partitioning. On the one hand, octree representation can eliminate the sparsity in the point cloud. On the other hand, in the voxel domain, convolutions can be naturally expres…
We present Hybrid Voxel Network (HVNet), a novel one-stage unified network for point cloud based 3D object detection for autonomous driving. Recent studies show that 2D voxelization with per voxel PointNet style feature extractor leads to accurate and efficient detector for large 3D scenes. Since the size of the feature map determines the computation and memory cost, the size of the voxel becomes a parameter that is hard to balance. A smaller voxel size gives a better performance, especially for small objects, but a longer inference time. A larger voxel can cover the same area with a smaller f…
We address the problem of 3D object detection, that is, estimating 3D object bounding boxes from point clouds. 3D object detection methods exploit either voxel-based or point-based features to represent 3D objects in a scene. Voxel-based features are efficient to extract, while they fail to preserve fine-grained 3D structures of objects. Point-based features, on the other hand, represent the 3D structures more accurately, but extracting these features is computationally expensive. We introduce in this paper a novel single-stage 3D detection method having the merit of both voxel-based and point…
Fudan University（复旦大学）；Institute of Science and Technology for Brain-Inspired Intelligence, Fudan University（复旦大学类脑智能科学与技术研究院）；Huashan Hospital, Fudan University（复旦大学附属华山医院）
Small lesions in magnetic resonance imaging (MRI) images are crucial for clinical diagnosis of many kinds of diseases. However, the MRI quality can be easily degraded by various noise, which can greatly affect the accuracy of diagnosis of small lesion. Although some methods for denoising MR images have been proposed, task-specific denoising methods for improving the diagnosis confidence of small lesions are lacking. In this work, we propose a voxel-wise hybrid residual MLP-CNN model to denoise three-dimensional (3D) MR images with small lesions. We combine basic deep learning architecture, MLP…
NVIDIA（英伟达）；University of Toronto（多伦多大学）；Vector Institute（矢量研究所）
We introduce DMTet, a deep 3D conditional generative model that can synthesize high-resolution 3D shapes using simple user guides such as coarse voxels. It marries the merits of implicit and explicit 3D representations by leveraging a novel hybrid 3D representation. Compared to the current implicit approaches, which are trained to regress the signed distance values, DMTet directly optimizes for the reconstructed surface, which enables us to synthesize finer geometric details with fewer artifacts. Unlike deep 3D generative models that directly generate explicit representations such as meshes, o…
Automatic liver lesion segmentation is a challenging task while having a significant impact on assisting medical professionals in the designing of effective treatment and planning proper care. In this paper we propose a cascaded system that combines both 2D and 3D convolutional neural networks to effectively segment hepatic lesions. Our 2D network operates on a slice by slice basis to segment the liver and larger tumors, while we use a 3D network to detect small lesions that are often missed in a 2D segmentation design. We employ this algorithm on the LiTS challenge obtaining a Dice score per …
The Chinese University of Hong Kong（香港中文大学）；SenseTime Research（商汤科技研究院）；Zhejiang University（浙江大学）；The University of Sydney（悉尼大学）；Nanyang Technological University（南洋理工大学）
Cascade is a classic yet powerful architecture that has boosted performance on various tasks. However, how to introduce cascade to instance segmentation remains an open question. A simple combination of Cascade R-CNN and Mask R-CNN only brings limited gain. In exploring a more effective approach, we find that the key to a successful instance segmentation cascade is to fully leverage the reciprocal relationship between detection and segmentation. In this work, we propose a new framework, Hybrid Task Cascade (HTC), which differs in two important aspects: (1) instead of performing cascaded refine…
National Key Laboratory of Science and Technology on Multi-spectral Information Processing（多光谱信息处理科学与技术国家重点实验室）；School of Artifical Intelligence and Automation（人工智能与自动化学院）；Huazhong University of Science and Technology（华中科技大学）
In this paper, we are committed to designing a high-performance network for two-view geometry. We first propose a Guided Loss and theoretically establish the direct negative correlation between the loss and Fn-measure by dynamically adjusting the weights of positive and negative classes during training, so that the network is always trained towards the direction of increasing Fn-measure. By this way, the network can maintain the advantage of the cross-entropy loss while maximizing the Fn-measure. We then propose a hybrid attention block to extract feature, which integrates the bayesian attenti…
Nagoya University（名古屋大学）；Gifu University（岐阜大学）；Nagoya University Graduate School of Medicine（名古屋大学医学研究生院）；Aichi Cancer Center（爱知癌症中心）
Recent advances in 3D fully convolutional networks (FCN) have made it feasible to produce dense voxel-wise predictions of volumetric images. In this work, we show that a multi-class 3D FCN trained on manually labeled CT scans of several anatomical structures (ranging from the large organs to thin vessels) can achieve competitive segmentation results, while avoiding the need for handcrafting features or training class-specific models. To this end, we propose a two-stage, coarse-to-fine approach that will first use a 3D FCN to roughly define a candidate region, which will then be used as input t…
FSHNet: Fully Sparse Hybrid Network for 3D Object Detection
School of Computer Science and Engineering, Sun Yat-sen University（计算机科学与工程学院，中山大学）；Key Laboratory of Machine Intelligence and Advanced Computing, Sun Yat-sen University（机器智能与高级计算关键实验室，中山大学）；School of Artificial Intelligence, Sun Yat-sen University（人工智能学院，中山大学）
Advanced Object Detection and Pose Estimation with Hybrid Task Cascade and High-Resolution Networks
California Institute of Technology（加州理工学院）；University of Michigan（密歇根大学）；University of Washington（华盛顿大学）；Washington University in St. Louis（圣路易斯华盛顿大学）；Northern Arizona University（北亚利桑那大学）
University of Science and Technology of China（中国科学技术大学）；JD AI Research（京东人工智能研究院）
Recent progress on 2D object detection has featured Cascade RCNN, which capitalizes on a sequence of cascade detectors to progressively improve proposal quality, towards high-quality object detection. However, there has not been evidence in support of building such cascade structures for 3D object detection, a challenging detection scenario with highly sparse LiDAR point clouds. In this work, we present a simple yet effective cascade architecture, named 3D Cascade RCNN, that allocates multiple detectors based on the voxelized point clouds in a cascade paradigm, pursuing higher quality 3D objec…
Huawei Noah’s Ark Lab（华为诺亚方舟实验室）；South China University of Technology（华南理工大学）
Object detectors trained on fully-annotated data currently yield state of the art performance but require expensive manual annotations. On the other hand, weakly-supervised detectors have much lower performance and cannot be used reliably in a realistic setting. In this paper, we study the hybrid-supervised object detection problem, aiming to train a high quality detector with only a limited amount of fullyannotated data and fully exploiting cheap data with imagelevel labels. State of the art methods typically propose an iterative approach, alternating between generating pseudo-labels and upda…
National Key Laboratory of Science and Technology on Multispectral Information Processing（多光谱信息处理科学与技术国家重点实验室）
Finding good correspondences is a critical prerequisite in many feature based tasks. Given a putative correspondence set of an image pair, we propose a neural network which finds correct correspondences by a binary-class classifier and estimates relative pose through classified correspondences. First, we analyze that due to the imbalance in the number of correct and wrong correspondences, the loss function has a great impact on the classification results. Thus, we propose a new Guided Loss that can directly use evaluation criterion (Fn-measure) as guidance to dynamically adjust the objective f…
Accurate diagnosis is required before performing proper treatments for coronary heart disease. Machine learning based approaches have been proposed by many researchers to improve the accuracy of coronary heart disease diagnosis. Ensemble learning and cascade generalization are among the methods which can be used to improve the generalization ability of learning algorithm. The objective of this study is to develop heart disease diagnosis method based on ensemble learning and cascade generalization. Cascade generalization method with loose coupling strategy is proposed in this study. C4. 5 and R…
University of Chinese Academy of Sciences（中国科学院大学）；Damo Academy, Alibaba Group（阿里巴巴集团达摩院）
LiDAR semantic segmentation essential for advanced autonomous driving is required to be accurate, fast, and easy-deployed on mobile platforms. Previous point-based or sparse voxel-based methods are far away from real-time applications since time-consuming neighbor searching or sparse 3D convolution are employed. Recent 2D projection-based methods, including range view and multi-view fusion, can run in real time, but suffer from lower accuracy due to information loss during the 2D projection. Besides, to improve the performance, previous methods usually adopt test time augmentation (TTA), which…
We introduce Hybrid Bayesian Eigenobjects (HBEOs), a novel representation for 3D objects designed to allow a robot to jointly estimate the pose, class, and full 3D geometry of a novel object observed from a single viewpoint in a single practical framework. By combining both linear subspace methods and deep convolutional prediction, HBEOs efficiently learn nonlinear object representations without directly regressing into high-dimensional space. HBEOs also remove the onerous and generally impractical necessity of input data voxelization prior to inference. We experimentally evaluate the suitabil…
University of New South Wales（新南威尔士大学）；Australian National University（澳大利亚国立大学）；School of Minerals and Energy Resources Engineering（矿物与能源资源工程学院）
Segmentation of 3D micro-Computed Tomographic uCT) images of rock samples is essential for further Digital Rock Physics (DRP) analysis, however, conventional methods such as thresholding, watershed segmentation, and converging active contours are susceptible to user-bias. Deep Convolutional Neural Networks (CNNs) have produced accurate pixelwise semantic segmentation results with natural images and $μ$CT rock images, however, physical accuracy is not well documented. The performance of 4 CNN architectures is tested for 2D and 3D cases in 10 configurations. Manually segmented uCT images of Mt. …
Eindhoven University of Technology（埃因霍温理工大学）；Shandong University of Science and Technology（山东科技大学）；Philips Research（飞利浦研究院）
Catheter segmentation in 3D ultrasound is important for computer-assisted cardiac intervention. However, a large amount of labeled images are required to train a successful deep convolutional neural network (CNN) to segment the catheter, which is expensive and time-consuming. In this paper, we propose a novel catheter segmentation approach, which requests fewer annotations than the supervised learning method, but nevertheless achieves better performance. Our scheme considers a deep Q learning as the pre-localization step, which avoids voxel-level annotation and which can efficiently localize t…
Facial Expression Recognition is a vital research topic in most fields ranging from artificial intelligence and gaming to Human-Computer Interaction (HCI) and Psychology. This paper proposes a hybrid model for Facial Expression recognition, which comprises a Deep Convolutional Neural Network (DCNN) and Haar Cascade deep learning architectures. The objective is to classify real-time and digital facial images into one of the seven facial emotion categories considered. The DCNN employed in this research has more convolutional layers, ReLU Activation functions, and multiple kernels to enhance filt…
University of Southern Denmark（南丹麦大学）；Newtec Engineering A/S（Newtec工程公司）；Mads Clausen Institute（马德斯·克劳森研究所）
We present a simple but novel hybrid approach to hyperspectral data cube reconstruction from computed tomography imaging spectrometry (CTIS) images that sequentially combines neural networks and the iterative Expectation Maximization (EM) algorithm. We train and test the ability of the method to reconstruct data cubes of $100\times100\times25$ and $100\times100\times100$ voxels, corresponding to 25 and 100 spectral channels, from simulated CTIS images generated by our CTIS simulator. The hybrid approach utilizes the inherent strength of the Convolutional Neural Network (CNN) with regard to noi…
