Due to the high similarity of disparity between consecutive frames in video sequences, the area where disparity changes is defined as the residual map, which can be calculated. Based on this, we propose RecSM, a network based on residual estimation with a flexible recursive structure for video stereo matching. The RecSM network accelerates stereo matching using a Multi-scale Residual Estimation Module (MREM), which employs the temporal context as a reference and rapidly calculates the disparity for the current frame by computing only the residual values between the current and previous frames. To further reduce the error of estimated disparities, we use the Disparity Optimization Module (DOM) and Temporal Attention Module (TAM) to enforce constraints between each module, and together with MREM, form a flexible Stackable Computation Structure (SCS), which allows for the design of different numbers of SCS based on practical scenarios. Experimental results demonstrate that with a stack count of 3, RecSM achieves a 4x speed improvement compared to ACVNet, running at 0.054 seconds based on one NVIDIA RTX 2080TI GPU, with an accuracy decrease of only 0.7%. Code is available at https://github.com/Y0uchenZ/RecSM.
Under stereo settings, the problem of image super-resolution (SR) and disparity estimation are interrelated that the result of each problem could help to solve the other. The effective exploitation of correspondence between different views facilitates the SR performance, while the high-resolution (HR) features with richer details benefit the correspondence estimation. According to this motivation, we propose a Stereo Super-Resolution and Disparity Estimation Feedback Network (SSRDE-FNet), which simultaneously handles the stereo image super-resolution and disparity estimation in a unified frame…
Estimating the confidence of disparity maps inferred by a stereo algorithm has become a very relevant task in the years, due to the increasing number of applications leveraging such cue. Although self-supervised learning has recently spread across many computer vision tasks, it has been barely considered in the field of confidence estimation. In this paper, we propose a flexible and lightweight solution enabling self-adapting confidence estimation agnostic to the stereo algorithm or network. Our approach relies on the minimum information available in any stereo setup (i.e., the input stereo pa…
We present a large-scale stereo RGB image object pose estimation dataset named the $\textbf{StereOBJ-1M}$ dataset. The dataset is designed to address challenging cases such as object transparency, translucency, and specular reflection, in addition to the common challenges of occlusion, symmetry, and variations in illumination and environments. In order to collect data of sufficient scale for modern deep learning models, we propose a novel method for efficiently annotating pose data in a multi-view fashion that allows data capturing in complex and flexible environments. Fully annotated with 6D …
Hadamard Attention Recurrent Transformer: A Strong Baseline for Stereo Matching Transformer
College of Computer Science, the State Key Laboratory of Public Big Data（计算机科学学院、公共大数据国家重点实验室）；School of Information Engineering（信息工程学院）；School of Software（软件学院）；Key Laboratory of Integrated Microsystems, Shenzhen Graduate School（集成微系统重点实验室、深圳研究生院）；School of Computer Science and Engineering（计算机科学与工程学院）
EgoEV-HandPose: Egocentric 3D Hand Pose Estimation and Gesture Recognition with Stereo Event Cameras
National Research Center for Optical Instrumentation, Zhejiang University（浙江大学光学仪器国家研究中心）；School of Artificial Intelligence and Robotics and the National Engineering Research Center of Robot Visual Perception and Control Technology, Hunan University（湖南大学人工智能与机器人学院和机器人视觉感知与控制技术国家工程研究中心）；Ant Group Company Ltd.（蚂蚁集团有限公司）
TiS-TSL: Image-Label Supervised Surgical Video Stereo Matching via Time-Switchable Teacher-Student Learning
Wuhan National Laboratory for Optoelectronics, Huazhong University of Science and Technology（华中科技大学光电实验室）；Wuhan United Imaging Surgical Co., Ltd.（武汉联合影像手术有限公司）
University of British Columbia（不列颠哥伦比亚大学）；Google Research（谷歌研究院）；University of Toronto（多伦多大学）
We propose a novel framework for finding correspondences in images based on a deep neural network that, given two images and a query point in one of them, finds its correspondence in the other. By doing so, one has the option to query only the points of interest and retrieve sparse correspondences, or to query all points in an image and obtain dense mappings. Importantly, in order to capture both local and global priors, and to let our model relate between image regions using the most relevant among said priors, we realize our network using a transformer. At inference time, we apply our corres…
DERD-Net: Learning Depth from Event-based Ray Densities
Technische Universität Berlin（柏林技术大学）；Einstein Center Digital Future（爱因斯坦数字未来中心）；Robotics Institute（机器人研究所）；Science of Intelligence Excellence Cluster（智能科学卓越集群）
