Markerless motion capture has become an active field of research in computer vision in recent years. Its extensive applications are known in a great variety of fields, including computer animation, human motion analysis, biomedical research, virtual reality, and sports science. Estimating human posture has recently gained increasing attention in the computer vision community, but due to the depth of uncertainty and the lack of the synthetic datasets, it is a challenging task. Various approaches have recently been proposed to solve this problem, many of which are based on deep learning. They are primarily focused on improving the performance of existing benchmarks with significant advances, especially 2D images. Based on powerful deep learning techniques and recently collected real-world datasets, we explored a model that can predict the skeleton of an animation based solely on 2D images. Frames generated from different real-world datasets with synthesized poses using different body shapes from simple to complex. The implementation process uses DeepLabCut on its own dataset to perform many necessary steps, then use the input frames to train the model. The output is an animated skeleton for human movement. The composite dataset and other results are the "ground truth" of the deep model.
Human pose estimation is a very active research field, stimulated by its important applications in robotics, entertainment or health and sports sciences, among others. Advances in convolutional networks triggered noticeable improvements in 2D pose estimation, leading modern 3D markerless motion capture techniques to an average error per joint of 20 mm. However, with the proliferation of methods, it is becoming increasingly difficult to make an informed choice. Here, we review the leading human pose estimation methods of the past five years, focusing on metrics, benchmarks and method structures…
Tracking Any Point Methods for Markerless 3D Tissue Tracking in Endoscopic Stereo Images
Institute of Medical Technology and Intelligent Systems, Hamburg University of Technology（医学技术与智能系统研究所，汉堡技术大学）；Department of Otorhinolaryngology, Head and Neck Surgery and Oncology, University Medical Center Hamburg-Eppendorf（耳鼻喉头颈外科与肿瘤学部，汉堡艾本多夫大学医学中心）；Martini-Klinik Prostate Cancer Center, University Medical Center Hamburg-Eppendorf（马尔廷诊所前列腺癌中心，汉堡艾本多夫大学医学中心）；Department of Urology, University Medical Center Hamburg-Eppendorf（泌尿外科，汉堡艾本多夫大学医学中心）
本文提出一种利用2D Tracking Any Point网络结合两个CoTracker模型实现内窥镜立体图像无标记3D组织追踪的新方法，实验验证了其在复杂手术场景中的准确性与潜力。
Video-Based Markerless Motion Capture for Clinical and Rehabilitation Biomechanics: A PRISMA-ScR Scoping Review of Validated Architectures, Clinical Readiness, and Emerging Methods
Université Côte d'Azur（蔚蓝海岸大学）；Centre Hospitalier Universitaire de Nice（尼斯大学附属医院）；Institut Universitaire de France（法国大学研究院）
Solving the camera-to-robot pose is a fundamental requirement for vision-based robot control, and is a process that takes considerable effort and cares to make accurate. Traditional approaches require modification of the robot via markers, and subsequent deep learning approaches enabled markerless feature extraction. Mainstream deep learning methods only use synthetic data and rely on Domain Randomization to fill the sim-to-real gap, because acquiring the 3D annotation is labor-intensive. In this work, we go beyond the limitation of 3D annotations for real-world data. We propose an end-to-end …
Conventional computer-assisted orthopaedic navigation systems rely on the tracking of dedicated optical markers for patient poses, which makes the surgical workflow more invasive, tedious, and expensive. Visual tracking has recently been proposed to measure the target anatomy in a markerless and effortless way, but the existing methods fail under real-world occlusion caused by intraoperative interventions. Furthermore, such methods are hardware-specific and not accurate enough for surgical applications. In this paper, we propose a RGB-D sensing-based markerless tracking method that is robust a…
Markerless Multi-view 3D Human Pose Estimation: a survey
Instituto de Engenharia de Sistemas e Computadores, Tecnologia e Ciência (INESC TEC)（系统与计算机工程、技术与科学研究所（INESC TEC））；Faculdade de Engenharia da Universidade do Porto (FEUP)（波尔图大学工程学院（FEUP））；Faculdade de Ciências da Universidade do Porto (FCUP)（波尔图大学理学院（FCUP））
Urban Flood Mapping Using Satellite Synthetic Aperture Radar Data: A Review of Characteristics, Approaches and Datasets
Technical University of Munich（慕尼黑工业大学）；Vienna University of Technology（维也纳工业大学）；ETH Zurich（苏黎世联邦理工学院）；Wuhan University（武汉大学）；DiXplorer AG（4DiXplorer公司）；Luxembourg Institute of Science and Technology (LIST)（卢森堡科学技术研究院）
Commonly used human motion capture systems require intrusive attachment of markers that are visually tracked with multiple cameras. In this work we present an efficient and inexpensive solution to markerless motion capture using only a few Kinect sensors. Unlike the previous work on 3d pose estimation using a single depth camera, we relax constraints on the camera location and do not assume a co-operative user. We apply recent image segmentation techniques to depth images and use curriculum learning to train our system on purely synthetic data. Our method accurately localizes body parts withou…
Deep neural networks have gained tremendous importance in many computer vision tasks. However, their power comes at the cost of large amounts of annotated data required for supervised training. In this work we review and compare different techniques available in the literature to improve training results without acquiring additional annotated real-world data. This goal is mostly achieved by applying annotation-preserving transformations to existing data or by synthetically creating more data.
Universidad Católica del Maule（马乌莱天主教大学）；Research Center for Advanced Studies of Maule（马乌莱高级研究中心）；Indian Institute of Technology Delhi（印度理工学院德里分校）
With the recent success of computer vision and deep learning, remarkable progress has been achieved on automatic personal recognition using vein biometrics. However, collecting large-scale real-world training data for palm vein recognition has turned out to be challenging, mainly due to the noise and irregular variations included at the time of acquisition. Meanwhile, existing palm vein recognition datasets are usually collected under near-infrared light, lacking detailed annotations on attributes (e.g., pose), so the influences of different attributes on vein recognition have been poorly inve…
Universitá di Siena（锡耶纳大学）；Politecnico di Milano（米兰理工大学）
Due to the reduction of technological costs and the increase of satellites launches, satellite images are becoming more popular and easier to obtain. Besides serving benevolent purposes, satellite data can also be used for malicious reasons such as misinformation. As a matter of fact, satellite images can be easily manipulated relying on general image editing tools. Moreover, with the surge of Deep Neural Networks (DNNs) that can generate realistic synthetic imagery belonging to various domains, additional threats related to the diffusion of synthetically generated satellite images are emergin…
Identity Card Presentation Attack Detection: A Systematic Review
da/sec-Biometrics and Internet Security Research Group, Hochschule Darmstadt（da/sec生物识别与互联网安全研究组，达姆施塔特高等学院）；Universidad Autónoma de Manizales（自动化工学院）；Universidad de Caldas（卡利斯大学）
Steklov Institute of Mathematics at St. Petersburg（圣彼得堡斯捷克洛夫数学研究所）
Synthetic data is an increasingly popular tool for training deep learning models, especially in computer vision but also in other areas. In this work, we attempt to provide a comprehensive survey of the various directions in the development and application of synthetic data. First, we discuss synthetic datasets for basic computer vision problems, both low-level (e.g., optical flow estimation) and high-level (e.g., semantic segmentation), synthetic environments and datasets for outdoor and urban scenes (autonomous driving), indoor scenes (indoor navigation), aerial navigation, simulation enviro…
Technische Hochschule Ulm - Ulm University of Applied Sciences（乌尔姆应用技术大学）；Universitätsklinikum Ulm - Ulm University Medical Center（乌尔姆大学医学中心）
Since the beginning of the COVID-19 pandemic, researchers have developed deep learning models to classify COVID-19 induced pneumonia. As with many medical imaging tasks, the quality and quantity of the available data is often limited. In this work we train a deep learning model on publicly available COVID-19 image data and evaluate the model on local hospital chest X-ray data. The data has been reviewed and labeled by two radiologists to ensure a high quality estimation of the generalization capabilities of the model. Furthermore, we are using a Generative Adversarial Network to generate synth…
Leibniz Universität Hannover（汉诺威莱布尼茨大学）；Institute of Mechatronic Systems（机电系统研究所）
In microsurgery, lasers have emerged as precise tools for bone ablation. A challenge is automatic control of laser bone ablation with 4D optical coherence tomography (OCT). OCT as high resolution imaging modality provides volumetric images of tissue and foresees information of bone position and orientation (pose) as well as thickness. However, existing approaches for OCT based laser ablation control rely on external tracking systems or invasively ablated artificial landmarks for tracking the pose of the OCT probe relative to the tissue. This can be superseded by estimating the scene flow cause…
