In human-computer interaction, it is important to accurately estimate the hand pose especially fingertips. However, traditional approaches for fingertip localization mainly rely on depth images and thus suffer considerably from the noise and missing values. Instead of depth images, stereo images can also provide 3D information of hands and promote 3D hand pose estimation. There are nevertheless limitations on the dataset size, global viewpoints, hand articulations and hand shapes in the publicly available stereo-based hand pose datasets. To mitigate these limitations and promote further research on hand pose estimation from stereo images, we propose a new large-scale binocular hand pose dataset called THU-Bi-Hand, offering a new perspective for fingertip localization. In the THU-Bi-Hand dataset, there are 447k pairs of stereo images of different hand shapes from 10 subjects with accurate 3D location annotations of the wrist and five fingertips. Captured with minimal restriction on the range of hand motion, the dataset covers large global viewpoint space and hand articulation space. To better present the performance of fingertip localization on THU-Bi-Hand, we propose a novel scheme termed Bi-stream Pose Guided Region Ensemble Network (Bi-Pose-REN). It extracts more representative feature regions around joint points in the feature maps under the guidance of the previously estimated pose. The feature regions are integrated hierarchically according to the topology of hand joints to regress the refined hand pose. Bi-Pose-REN and several existing methods are evaluated on THU-Bi-Hand so that benchmarks are provided for further research. Experimental results show that our new method has achieved the best performance on THU-Bi-Hand.
SpikeStereoNet: A Brain-Inspired Framework for Stereo Depth Estimation from Spike Streams
National Key Laboratory for Multimedia Information Processing, School of Computer Science, Peking University(北京大学计算机学院多媒体信息处理全国重点实验室);Center on Frontiers of Computing Studies, School of Computer Science, Peking University(北京大学计算机学院计算前沿研究中心);Institute for Artificial Intelligence, Peking University(北京大学人工智能研究院);School of Engineering and Applied Science, University of Pennsylvania(宾夕法尼亚大学工程与应用科学学院)
PIE-PS: Photometric Stereo from Physical Irradiance Event Streams
Jilin University(吉林大学);Guangdong Institute of Intelligence Science and Technology(广东智能科学与技术研究院);Beijing Institute of Technology(北京理工大学);The Hong Kong University of Science and Technology(香港科技大学)
Online Adaptation through Meta-Learning for Stereo Depth Estimation
Nanjing University of Science and Technology(南京理工大学);University of Trento(特伦托大学);Huawei Technologies Ireland(华为技术爱尔兰公司);Fondazione Bruno Kessler(布鲁诺·凯斯勒基金会)
Retain, Blend, and Exchange: A Quality-aware Spatial-Stereo Fusion Approach for Event Stream Recognition
School of Electronic and Information Engineering, Anhui University(安徽大学电子与信息工程学院);School of Computer Science and Technology, Anhui University(安徽大学计算机科学与技术学院);Department of Automation, Tsinghua University(清华大学自动化系);Peng Cheng Laboratory(鹏城实验室);Harbin Institute of Technology (HITSZ)(哈尔滨工业大学(深圳));School of Computer Science, Peking University(北京大学计算机学院);School of Electronic and Computer Engineering, Shenzhen Graduate School, Peking University(北京大学深圳研究生院电子与计算机工程学院)
Eventor: An Efficient Event-Based Monocular Multi-View Stereo Accelerator on FPGA Platform
Beihang University(北京航空航天大学);Xidian University(西安电子科技大学);Beijing Real Imaging Medical Technology Co., Ltd.(北京真实影像医疗科技有限公司);The Chinese University of Hong Kong(香港中文大学)
EgoEV-HandPose: Egocentric 3D Hand Pose Estimation and Gesture Recognition with Stereo Event Cameras
National Research Center for Optical Instrumentation, Zhejiang University(浙江大学光学仪器国家研究中心);School of Artificial Intelligence and Robotics and the National Engineering Research Center of Robot Visual Perception and Control Technology, Hunan University(湖南大学人工智能与机器人学院和机器人视觉感知与控制技术国家工程研究中心);Ant Group Company Ltd.(蚂蚁集团有限公司)