Learning Photometric Feature Transform for Free-form Object Scan
State Key Lab of CAD & CG, Zhejiang University（浙江大学CAD&CG国家重点实验室）；Department of Electronic and Computer Engineering, The Hong Kong University of Science and Technology（香港科技大学电子与计算机工程系）；Visual Computing Center, King Abdullah University of Science and Technology（阿卜杜拉国王科技大学视觉计算中心）
Xiang Feng, Kaizhang Kang, Fan Pei, Huakeng Ding, Jinjiang You, Ping Tan, Kun Zhou, Hongzhi Wu
We propose a novel framework to automatically learn to aggregate and transform photometric measurements from multiple unstructured views into spatially distinctive and view-invariant low-level features, which are subsequently fed to a multi-view stereo pipeline to enhance 3D reconstruction. The illumination conditions during acquisition and the feature transform are jointly trained on a large amount of synthetic data. We further build a system to reconstruct both the geometry and anisotropic reflectance of a variety of challenging objects from hand-held scans. The effectiveness of the system is demonstrated with a lightweight prototype, consisting of a camera and an array of LEDs, as well as an off-the-shelf tablet. Our results are validated against reconstructions from a professional 3D scanner and photographs, and compare favorably with state-of-the-art techniques.
Technical University of Munich（慕尼黑工业大学）；Google（谷歌）
We present SPSG, a novel approach to generate high-quality, colored 3D models of scenes from RGB-D scan observations by learning to infer unobserved scene geometry and color in a self-supervised fashion. Our self-supervised approach learns to jointly inpaint geometry and color by correlating an incomplete RGB-D scan with a more complete version of that scan. Notably, rather than relying on 3D reconstruction losses to inform our 3D geometry and color reconstruction, we propose adversarial and perceptual losses operating on 2D renderings in order to achieve high-resolution, high-quality colored …
Learning Geometric and Photometric Features from Panoramic LiDAR Scans for Outdoor Place Categorization
Graduate School of Information Science and Electrical Engineering, Kyushu University（九州大学信息科学与电子工程研究生院）；Jet Propulsion Laboratory, California Institute of Technology（加州理工学院喷气推进实验室）；Faculty of Information Science and Electrical Engineering, Kyushu University（九州大学信息科学与电子工程学部）；Technical University of Cartagena（卡塔赫纳技术大学）
We present a novel framework to learn to convert the perpixel photometric information at each view into spatially distinctive and view-invariant low-level features, which can be plugged into existing multi-view stereo pipeline for enhanced 3D reconstruction. Both the illumination conditions during acquisition and the subsequent per-pixel feature transform can be jointly optimized in a differentiable fashion. Our framework automatically adapts to and makes efficient use of the geometric information available in different forms of input data. High-quality 3D reconstructions of a variety of chall…
National Institute of Technology, Calicut（卡利卡特国家理工学院）；Nanjing Forestry University（南京林业大学）；Saint Mary’s University（圣玛丽大学）
Rigid registration of multi-view and multi-platform LiDAR scans is a fundamental problem in 3D mapping, robotic navigation, and large-scale urban modeling applications. Data acquisition with LiDAR sensors involves scanning multiple areas from different points of view, thus generating partially overlapping point clouds of the real world scenes. Traditionally, ICP (Iterative Closest Point) algorithm is used to register the acquired point clouds together to form a unique point cloud that captures the scanned real world scene. Conventional ICP faces local minima issues and often needs a coarse ini…
Lidar data can be used to generate point clouds for the navigation of autonomous vehicles or mobile robotics platforms. Scan matching, the process of estimating the rigid transformation that best aligns two point clouds, is the basis for lidar odometry, a form of dead reckoning. Lidar odometry is particularly useful when absolute sensors, like GPS, are not available. Here we propose the Iterative Closest Ellipsoidal Transform (ICET), a scan matching algorithm which provides two novel improvements over the current state-of-the-art Normal Distributions Transform (NDT). Like NDT, ICET decomposes …
TSegFormer: 3D Tooth Segmentation in Intraoral Scans with Geometry Guided Transformer
ZJU-UIUC Institute（ZJU-UIUC联合学院）；Zhejiang University（浙江大学）；Stomatology Hospital（口腔医院）；School of Stomatology, Zhejiang University School of Medicine（浙江大学医学院口腔医学院）；Angelalign Research Institute（时代天使公司）；Institute of High Performance Computing (IHPC), A*STAR（新加坡科技研究局高性能计算研究所）；ChohoTech Inc.（初浩科技有限公司）；School of Public Health, Zhejiang University（浙江大学公共卫生学院）
Neural Field-Based 3D Surface Reconstruction of Microstructures from Multi-Detector Signals in Scanning Electron Microscopy
State Key Lab of CAD&CG, Zhejiang University（浙江大学CAD&CG国家重点实验室）；Alibaba Group（阿里巴巴集团）；Analysis Center of Agriculture, Life and Environment Sciences, Zhejiang University（浙江大学农业、生命与环境科学分析中心）
In this paper, we introduce SDM-UniPS, a groundbreaking Scalable, Detailed, Mask-free, and Universal Photometric Stereo network. Our approach can recover astonishingly intricate surface normal maps, rivaling the quality of 3D scanners, even when images are captured under unknown, spatially-varying lighting conditions in uncontrolled environments. We have extended previous universal photometric stereo networks to extract spatial-light features, utilizing all available information in high-resolution input images and accounting for non-local interactions among surface points. Moreover, we present…
Johns Hopkins University（约翰斯·霍普金斯大学）；University of California San Diego（加利福尼亚大学圣迭戈分校）；Tongji University（同济大学）；The Johns Hopkins University School of Medicine（约翰斯·霍普金斯大学医学院）
Tubular structure segmentation in medical images, e.g., segmenting vessels in CT scans, serves as a vital step in the use of computers to aid in screening early stages of related diseases. But automatic tubular structure segmentation in CT scans is a challenging problem, due to issues such as poor contrast, noise and complicated background. A tubular structure usually has a cylinder-like shape which can be well represented by its skeleton and cross-sectional radii (scales). Inspired by this, we propose a geometry-aware tubular structure segmentation method, Deep Distance Transform (DDT), which…
Vision Transformer for Intracranial Hemorrhage Classification in CT Scans Using an Entropy-Aware Fuzzy Integral Strategy for Adaptive Scan-Level Decision Fusion
School of Electrical and Computer Engineering, College of Engineering, University of Tehran（塔里斯坦大学电气与计算机工程学院）；Department of Computer Science and Engineering, Sejong University（世宗大学计算机科学与工程系）
With the massive damage in the world caused by Coronavirus Disease 2019 SARS-CoV-2 (COVID-19), many related research topics have been proposed in the past two years. The Chest Computed Tomography (CT) scans are the most valuable materials to diagnose the COVID-19 symptoms. However, most schemes for COVID-19 classification of Chest CT scan is based on a single-slice level, implying that the most critical CT slice should be selected from the original CT scan volume manually. We simultaneously propose 2-D and 3-D models to predict the COVID-19 of CT scan to tickle this issue. In our 2-D model, we…
In the following paper, we have combined the various basic functionalities provided by the NumPy library and OpenCv library, which is an open source for Computer Vision applications, like conversion of colored images to grayscale, calculating threshold, finding contours and using those contour points to take perspective transform of the image inputted by the user, using Python version 3.7. Additional features include cropping, rotating and saving as well. All these functions and features, when implemented step by step, results in a complete scanning application. The applied procedure involves …
University of Waterloo（滑铁卢大学）；Neptec Design Group（Neptec设计集团）
Topographic mapping in planetary environments relies on accurate 3D scan registration methods. However, most global registration algorithms relying on features such as FPFH and Harris-3D show poor alignment accuracy in these settings due to the poor structure of the Mars-like terrain and variable resolution, occluded, sparse range data that is hard to register without some a-priori knowledge of the environment. In this paper, we propose an alternative approach to 3D scan registration using the curvelet transform that performs multi-resolution geometric analysis to obtain a set of coefficients …
Computed tomography (CT) imaging could be very practical for diagnosing various diseases. However, the nature of the CT images is even more diverse since the resolution and number of the slices of a CT scan are determined by the machine and its settings. Conventional deep learning models are hard to tickle such diverse data since the essential requirement of the deep neural network is the consistent shape of the input data. In this paper, we propose a novel, effective, two-step-wise approach to tickle this issue for COVID-19 symptom classification thoroughly. First, the semantic feature embedd…
