Measuring Generalisation to Unseen Viewpoints, Articulations, Shapes and Objects for 3D Hand Pose Estimation under Hand-Object Interaction
Imperial College London（帝国理工学院）；Niantic, Inc.（Niantic公司）；Graz Uni. of Technology（格拉茨工业大学）；Rokid Corp. Ltd.（Rokid公司）；HUST（华中科技大学）；Megvii Research Nanjing（旷视南京研究院）；SUNY Buffalo（纽约州立大学布法罗分校）；BUPT（北京邮电大学）；Uni. of West Bohemia（西波希米亚大学）；Fudan Uni.（复旦大学）；TUM（慕尼黑工业大学）；Uni. of Bonn（波恩大学）；NUS（新加坡国立大学）；Harbin Inst. of Technology（哈尔滨工业大学）；CUHK（香港中文大学）；ETH Zurich（苏黎世联邦理工学院）；NVIDIA Research（英伟达研究院）；NAVER LABS Europe（NAVER欧洲实验室）；ENPC ParisTech（巴黎高科路桥学校）；UNIST（蔚山科学技术院）；KAIST（韩国科学技术院）
We study how well different types of approaches generalise in the task of 3D hand pose estimation under single hand scenarios and hand-object interaction. We show that the accuracy of state-of-the-art methods can drop, and that they fail mostly on poses absent from the training set. Unfortunately, since the space of hand poses is highly dimensional, it is inherently not feasible to cover the whole space densely, despite recent efforts in collecting large-scale training datasets. This sampling problem is even more severe when hands are interacting with objects and/or inputs are RGB rather than depth images, as RGB images also vary with lighting conditions and colors. To address these issues, we designed a public challenge (HANDS'19) to evaluate the abilities of current 3D hand pose estimators (HPEs) to interpolate and extrapolate the poses of a training set. More exactly, HANDS'19 is designed (a) to evaluate the influence of both depth and color modalities on 3D hand pose estimation, under the presence or absence of objects; (b) to assess the generalisation abilities w.r.t. four main axes: shapes, articulations, viewpoints, and objects; (c) to explore the use of a synthetic hand model to fill the gaps of current datasets. Through the challenge, the overall accuracy has dramatically improved over the baseline, especially on extrapolation tasks, from 27mm to 13mm mean joint error. Our analyses highlight the impacts of: Data pre-processing, ensemble approaches, the use of a parametric 3D hand model (MANO), and different HPE methods/backbones.
The 2017 Hands in the Million Challenge on 3D Hand Pose Estimation
Imperial College London（帝国理工学院）
我们提出了2017年Hands in the Million Challenge，这是一项旨在评估3D手部姿势估计任务的公开竞赛。这项挑战的目标是评估在解决3D手部姿势估计问题方面的最新技术水平，以及检测系统和评估指标的主要故障和强度模式，以帮助确定未来的研究方向。该挑战是最近发布的BigHand2.2M和第一人称手部动作数据集的后续，这些数据集旨在详尽地涵盖多个手部、视点、手部关节和遮挡。该挑战赛包括一个标准化的数据集，两个不同任务的评估协议，以及一个公开的比赛。在本文件中，我们描述了挑战的不同方面，并与参与者的结果一起，将在2017年第三届观察和理解动手国际研讨会（HANDS 2017）上与ICCV 2017一起展示。
Joint Multi-Condition Representation Modelling via Matrix Factorisation for Visual Place Recognition
School of Electronics and Computer Science, University of Southampton（苏塞克斯大学电子与计算机科学学院）；Department of Computer Science and Engineering, Hanyang University（翰阳大学计算机科学与工程系）；QUT Centre for Robotics, School of Electrical Engineering and Robotics（昆士兰大学机器人中心、电气工程与机器人学院）；School of Computer Science and Electronic Engineering, University of Essex（埃塞克斯大学计算机科学与电子工程学院）
In this article we consider a version of the challenging problem of learning from datasets whose size is too limited to allow generalisation beyond the training set. To address the challenge we propose to use a transfer learning approach whereby the model is first trained on a synthetic dataset replicating features of the original objects. In this study the objects were smartphone photographs of near-complete Roman terra sigillata pottery vessels from the collection of the Museum of London. Taking the replicated features from published profile drawings of pottery forms allowed the integration …
University of Surrey（萨里大学）；Vicon Motion Systems Ltd（维康运动系统有限公司）
We present a novel method to improve the accuracy of the 3D reconstruction of clothed human shape from a single image. Recent work has introduced volumetric, implicit and model-based shape learning frameworks for reconstruction of objects and people from one or more images. However, the accuracy and completeness for reconstruction of clothed people is limited due to the large variation in shape resulting from clothing, hair, body size, pose and camera viewpoint. This paper introduces two advances to overcome this limitation: firstly a new synthetic dataset of realistic clothed people, 3DVH; an…
University of Zagreb, Faculty of Electrical Engineering and Computing（萨格勒布大学电气工程与计算学院）
In this paper we propose a novel approach to generate a synthetic aerial dataset for application in UAV monitoring. We propose to accentuate shape-based object representation by applying texture randomization. A diverse dataset with photorealism in all parameters such as shape, pose, lighting, scale, viewpoint, etc. except for atypical textures is created in a 3D modelling software Blender. Our approach specifically targets two conditions in aerial images where texture of objects is difficult to detect, namely challenging illumination and objects occupying only a small portion of the image. Ex…
