Recovering the 3D structure of the surrounding environment is an essential task in any vision-controlled Structure-from-Motion (SfM) scheme. This paper focuses on the theoretical properties of the SfM, known as the incremental active depth estimation. The term incremental stands for estimating the 3D structure of the scene over a chronological sequence of image frames. Active means that the camera actuation is such that it improves estimation performance. Starting from a known depth estimation filter, this paper presents the stability analysis of the filter in terms of the control inputs of the camera. By analyzing the convergence of the estimator using the Lyapunov theory, we relax the constraints on the projection of the 3D point in the image plane when compared to previous results. Nonetheless, our method is capable of dealing with the cameras' limited field-of-view constraints. The main results are validated through experiments with simulated data.
ASGrasp: Generalizable Transparent Object Reconstruction and 6-DoF Grasp Detection from RGB-D Active Stereo Camera
Samsung R&D Institute China-Beijing（三星中国北京研发中心）；CFCS, School of Computer Science, Peking University（计算机科学系，北京大学）；Beijing Academy of Artificial Intelligence (BAAI)（北京人工智能研究院）
An active-learning framework for real-time depth perception from monocular vision streams
School of Mechanical and Automotive Engineering, Xiamen University of Technology（厦门理工学院机械与汽车工程学院）；Xiamen Innovative Centre for Automotive Electric Driving（厦门新能源汽车驱动创新中心）；King Long United Automotive Industry Co. Ltd.（金龙联合汽车工业有限公司）；School of Electrical and Electronic Engineering, The University of Adelaide（阿德莱德大学电气与电子工程学院）；School of Vehicle and Mobility, Tsinghua University（清华大学车辆与运载学院）
University of Toronto Institute for Aerospace Studies（多伦多大学航空航天研究所）；Robotics Institute（机器人研究所）
Depth acquisition with the active stereo camera is a challenging task for highly reflective objects. When setup permits, multi-view fusion can provide increased levels of depth completion. However, due to the slow acquisition speed of high-end active stereo cameras, collecting a large number of viewpoints for a single scene is generally not practical. In this work, we propose a next-best-view framework to strategically select camera viewpoints for completing depth data on reflective objects. In particular, we explicitly model the specular reflection of reflective surfaces based on the Phong re…
Towards Ultrafast Depth Sensing Via Active Event-based Stereo Vision
School of Computer Science, Peking University（北京大学计算机科学学院）；Peng Cheng Laboratory（鹏城实验室）；Department of Automation, Tsinghua University（清华大学自动化系）；Department of Computer Science and Technology, Tsinghua University（清华大学计算机科学与技术系）
University of Toronto Institute for Aerospace Studies（多伦多大学航空航天研究所）；Robotics Institute（机器人研究所）；Epson Canada Ltd.（爱普生加拿大有限公司）
The reliable fusion of depth maps from multiple viewpoints has become an important problem in many 3D reconstruction pipelines. In this work, we investigate its impact on robotic bin-picking tasks such as 6D object pose estimation. The performance of object pose estimation relies heavily on the quality of depth data. However, due to the prevalence of shiny surfaces and cluttered scenes, industrial grade depth cameras often fail to sense depth or generate unreliable measurements from a single viewpoint. To this end, we propose a novel probabilistic framework for scene reconstruction in robotic …
University of California, San Diego（加利福尼亚大学圣迭戈分校）；Tsinghua University（清华大学）
In this paper, we focus on the simulation of active stereovision depth sensors, which are popular in both academic and industry communities. Inspired by the underlying mechanism of the sensors, we designed a fully physics-grounded simulation pipeline that includes material acquisition, ray-tracing-based infrared (IR) image rendering, IR noise simulation, and depth estimation. The pipeline is able to generate depth maps with material-dependent error patterns similar to a real depth sensor in real time. We conduct real experiments to show that perception algorithms and reinforcement learning pol…
Robotics Institute, School of Computer Science, Carnegie Mellon University（卡内基梅隆大学计算机科学学院机器人研究所）；School of Earth Sciences, Ohio State University（俄亥俄州立大学地球科学学院）
The active search for objects of interest in an unknown environment has many robotics applications including search and rescue, detecting gas leaks or locating animal poachers. Existing algorithms often prioritize the location accuracy of objects of interest while other practical issues such as the reliability of object detection as a function of distance and lines of sight remain largely ignored. Additionally, in many active search scenarios, communication infrastructure may be unreliable or unestablished, making centralized control of multiple agents impractical. We present an algorithm call…
University of Connecticut（康涅狄格大学）；Oklahoma State University（俄克拉荷马州立大学）
In this paper concurrent learning (CL)-based full and reduced order observers for a perspective dynamical system (PDS) are developed. The PDS is a widely used model for estimating the depth of a feature point from a sequence of camera images. Building on the current progress of CL for parameter estimation in adaptive control, a state observer is developed for the PDS model where the inverse depth appears as a time-varying parameter in the dynamics. The data recorded over a sliding time window in the near past is used in the CL term to design the full and the reduced order state observers. A Ly…
