Photometric stereo provides an important method for high-fidelity 3D reconstruction based on multiple intensity images captured under different illumination directions. In this paper, we present a complete framework, including a multilight source illumination and acquisition hardware system and a two-stage convolutional neural network (CNN) architecture, to construct inter- and intraframe representations for accurate normal estimation of non-Lambertian objects. We experimentally investigate numerous network design alternatives for identifying the optimal scheme to deploy inter- and intraframe feature extraction modules for the photometric stereo problem. Moreover, we propose utilizing the easily obtained object mask to eliminate adverse interference from invalid background regions in intraframe spatial convolutions, thus effectively improving the accuracy of normal estimation for surfaces made of dark materials or with cast shadows. Experimental results demonstrate that the proposed masked two-stage photometric stereo CNN model (MT-PS-CNN) performs favourably against state-of-the-art photometric stereo techniques in terms of both accuracy and efficiency. In addition, the proposed method is capable of predicting accurate and rich surface normal details for non-Lambertian objects of complex geometry and performs stably given inputs captured in both sparse and dense lighting distributions.
Stereo Anywhere: Robust Zero-Shot Deep Stereo Matching Even Where Either Stereo or Mono Fail
Advanced Research Center on Electronic System (ARCES)（电子系统高级研究中心）；Department of Computer Science and Engineering (DISI)（计算机科学与工程系）；University of Bologna（博洛尼亚大学）
Diving into the Fusion of Monocular Priors for Generalized Stereo Matching
Beijing Key Laboratory of Intelligent Information Technology, School of Computer Science & Technology, Beijing Institute of Technology, China（北京智能信息科技重点实验室，计算机科学与技术学院，北京理工大学，中国）；Guangdong Laboratory of Machine Perception and Intelligent Computing, Shenzhen MSU-BIT University, China（广东机器感知与智能计算实验室，深圳MSU-BIT大学，中国）；NVIDIA
Geometry Reinforced Efficient Attention Tuning Equipped with Normals for Robust Stereo Matching
Department of Computer Science, City University of Hong Kong（香港城市大学计算机科学系）；Department of Computer Science, Southern Methodist University（南卫理公会大学计算机科学系）；Hon Hai Research Institute（鸿海研究院）
Deep Learning Methods for Calibrated Photometric Stereo and Beyond
Nanyang Technological University（南洋理工大学）；The Hong Kong Polytechnic University（香港理工大学）；Shenzhen University（深圳大学）；University of Leicester（莱斯特大学）；Ocean University of China（中国海洋大学）；Peking University（北京大学）
KeyPose: Multi-View 3D Labeling and Keypoint Estimation for Transparent Objects
Stanford University（斯坦福大学）；Robotics at Google（谷歌机器人部门）
估计桌面对象的3D姿态对于机器人操作等应用至关重要。许多现有的解决这个问题的方法都需要对象的深度图来进行训练和预测，这将它们限制在不透明的朗伯对象上，这些对象在RGBD传感器中产生良好的回报。在本文中，我们放弃使用深度传感器，而采用原始立体声输入。我们解决两个问题：首先，我们建立了一种简单的方法，用于使用RGB相机捕获和标记桌面对象上的3D关键点;其次，我们开发了一种名为$KeyPose$的深度神经网络，该网络可以学习使用3D关键点准确预测对象姿势，从立体输入，甚至适用于透明对象。为了评估我们的方法的性能，我们创建了一个包含五个类别的15个清晰对象的数据集，其中包含48 K 3D关键点标记的图像。我们训练实例和类别模型，并展示对新纹理、姿势和对象的泛化。KeyPose在此数据集上的3D姿态估计性能超过最先进的性能1.5至3.5倍，即使在竞争方法提供地面实况深度的情况下也是如此。立体声输入对于此性能至关重要，因为与使用单目输入相比，它可以将结果提高2倍。我们将发布数据捕获和标签管道的公共版本，透明对象数据库，以及KeyPose模型和评估代码。项目网址：https://sites.google.com/corp/view/keypose。
