In this paper, we focus on the simulation of active stereovision depth sensors, which are popular in both academic and industry communities. Inspired by the underlying mechanism of the sensors, we designed a fully physics-grounded simulation pipeline that includes material acquisition, ray-tracing-based infrared (IR) image rendering, IR noise simulation, and depth estimation. The pipeline is able to generate depth maps with material-dependent error patterns similar to a real depth sensor in real time. We conduct real experiments to show that perception algorithms and reinforcement learning policies trained in our simulation platform could transfer well to the real-world test cases without any fine-tuning. Furthermore, due to the high degree of realism of this simulation, our depth sensor simulator can be used as a convenient testbed to evaluate the algorithm performance in the real world, which will largely reduce the human effort in developing robotic algorithms. The entire pipeline has been integrated into the SAPIEN simulator and is open-sourced to promote the research of vision and robotics communities.
Towards Ultrafast Depth Sensing Via Active Event-based Stereo Vision
School of Computer Science, Peking University（北京大学计算机科学学院）；Peng Cheng Laboratory（鹏城实验室）；Department of Automation, Tsinghua University（清华大学自动化系）；Department of Computer Science and Technology, Tsinghua University（清华大学计算机科学与技术系）
D3RoMa: Disparity Diffusion-based Depth Sensing for Material-Agnostic Robotic Manipulation
CFCS；School of Computer Science, Peking University（北京大学计算机学院）；Galbot；University of California, Berkeley（加州大学伯克利分校）；Stanford University（斯坦福大学）；University of Chinese Academy of Sciences（中国科学院大学）；Beijing Academy of Artificial Intelligence（北京人工智能研究院）
Calibration-Free Surface Normals Estimation in Vision-Based Tactile Sensing using Universal Photometric Stereo
TU Dresden（德累斯顿工业大学）；TUD Dresden University of Technology（德累斯顿工业大学）；Centre for Tactile Internet with Human-in-the-Loop (CeTI)（人在环触觉互联网中心（CeTI））；DKFZ（德国癌症研究中心）；University Hospital Carl Gustav Carus（卡尔·古斯塔夫·卡鲁斯大学医院）；Helmholtz-Zentrum Dresden-Rossendorf (HZDR)（德累斯顿-罗森多夫亥姆霍兹中心（HZDR））
SymmeTac: Symmetric Color LED Driven Efficient Photometric Stereo Reconstruction Methods for Camera-based Tactile Sensors
School of Artificial Intelligence, Beijing University of Posts and Telecom.（北京邮电大学人工智能学院）；School of Mechanical Engineering, Shanghai Jiao Tong University（上海交通大学机械工程学院）；School of Computer Science, Peking University（北京大学计算机学院）
Fusing Laser Scanner and Stereo Camera in Evidential Grid Maps
Center of Robotics（机器人中心）；Mines ParisTech（巴黎高等矿业学院）；PSL Research University（巴黎文理研究大学）；California PATH（加州PATH研究所）；University of California, Berkeley（加州大学伯克利分校）
CompdVision: Combining Near-Field 3D Visual and Tactile Sensing Using a Compact Compound-Eye Imaging System
The Hong Kong University of Science and Technology（香港科技大学）；Great Bay University（大湾区大学）；HKUST Shenzhen-Hong Kong Collaborative Innovation Research Institute（香港科技大学深圳香港协同创新研究院）
Panoramic Depth Estimation via Supervised and Unsupervised Learning in Indoor Scenes
Zhejiang University（浙江大学）；Karlsruhe Institute of Technology（卡尔斯鲁厄理工学院）；National Engineering Research Center of Optical Instrumentation, Zhejiang University（浙江大学国家光学仪器工程技术研究中心）
