In this work, we investigate the influence of labeling methods on the classification of human movements on data recorded using a marker-based motion capture system. The dataset is labeled using two different approaches, one based on video data of the movements, the other based on the movement trajectories recorded using the motion capture system. The dataset is labeled using two different approaches, one based on video data of the movements, the other based on the movement trajectories recorded using the motion capture system. The data was recorded from one participant performing a stacking scenario comprising simple arm movements at three different speeds (slow, normal, fast). Machine learning algorithms that include k-Nearest Neighbor, Random Forest, Extreme Gradient Boosting classifier, Convolutional Neural networks (CNN), Long Short-Term Memory networks (LSTM), and a combination of CNN-LSTM networks are compared on their performance in recognition of these arm movements. The models were trained on actions performed on slow and normal speed movements segments and generalized on actions consisting of fast-paced human movement. It was observed that all the models trained on normal-paced data labeled using trajectories have almost 20% improvement in accuracy on test data in comparison to the models trained on data labeled using videos of the performed experiments.
Co-Speech Gesture Detection through Multi-Phase Sequence Labeling
University of Amsterdam（阿姆斯特丹大学）；ScaDS.AI Dresden/Leipzig（德累斯顿/莱比锡数据科学中心人工智能研究所）；TU Dresden（德累斯顿工业大学）；Meertens Institute（梅滕斯研究所）；Radboud University（拉德堡德大学）；Max Planck Institute for Psycholinguistics（马克斯·普朗克心理语言学研究所）
本文提出将协同语音手势检测重新构建为多阶段序列标注问题，通过Transformer编码器和Conditional Random Fields处理骨骼运动序列，显著提升了手势划动和手势单元检测性能。
This paper introduces the visually informed embedding of word (VIEW), a continuous vector representation for a word extracted from a deep neural model trained using the Microsoft COCO data set to forecast the spatial arrangements between visual objects, given a textual description. The model is composed of a deep multilayer perceptron (MLP) stacked on the top of a Long Short Term Memory (LSTM) network, the latter being preceded by an embedding layer. The VIEW is applied to transferring multimodal background knowledge to Spatial Role Labeling (SpRL) algorithms, which recognize spatial relations…
Unsupervised domain adaptation aims to address the problem of classifying unlabeled samples from the target domain whilst labeled samples are only available from the source domain and the data distributions are different in these two domains. As a result, classifiers trained from labeled samples in the source domain suffer from significant performance drop when directly applied to the samples from the target domain. To address this issue, different approaches have been proposed to learn domain-invariant features or domain-specific classifiers. In either case, the lack of labeled samples in the…
Mesh labeling is the key problem of classifying the facets of a 3D mesh with a label among a set of possible ones. State-of-the-art methods model mesh labeling as a Markov Random Field over the facets. These algorithms map image segmentations to the mesh by minimizing an energy function that comprises a data term, a smoothness terms, and class-specific priors. The latter favor a labeling with respect to another depending on the orientation of the facet normals. In this paper we propose a novel energy term that acts as a prior, but does not require any prior knowledge about the scene nor scene-…
Gdańsk University of Technology（格但斯克理工大学）；AI Sweden（瑞典人工智能）；SiMa Technologies（SiMa科技公司）；VoiceLab, NLP Lab（VoiceLab自然语言处理实验室）；Intel Corporation（英特尔公司）；Silesian University of Technology（西里西亚理工大学）
Labeling is the cornerstone of supervised machine learning, which has been exploited in a plethora of various applications, with sign language recognition being one of them. However, such algorithms must be fed with a huge amount of consistently labeled data during the training process to elaborate a well-generalizing model. In addition, there is a great need for an automated solution that works with any nationally diversified sign language. Although there are language-agnostic transcription systems, such as the Hamburg Sign Language Notation System (HamNoSys) that describe the signer's initia…
Indian Institute of Technology Kanpur（印度理工学院坎普尔分校）；ETALYC Inc.（ETALYC公司）；City of Dubuque（迪比克市）；Iowa State University（爱荷华州立大学）
Movement specific vehicle classification and counting at traffic intersections is a crucial component for various traffic management activities. In this context, with recent advancements in computer-vision based techniques, cameras have emerged as a reliable data source for extracting vehicular trajectories from traffic scenes. However, classifying these trajectories by movement type is quite challenging as characteristics of motion trajectories obtained this way vary depending on camera calibrations. Although some existing methods have addressed such classification tasks with decent accuracie…
IMT Lille Douai（里尔杜埃IMT商学院）；Univ. Lille（里尔大学）；CNRS（法国国家科学研究中心）；University of Florence（佛罗伦萨大学）
Emotion recognition is attracting great interest for its potential application in a multitude of real-life situations. Much of the Computer Vision research in this field has focused on relating emotions to facial expressions, with investigations rarely including more than upper body. In this work, we propose a new scenario, for which emotional states are related to 3D dynamics of the whole body motion. To address the complexity of human body movement, we used covariance descriptors of the sequence of the 3D skeleton joints, and represented them in the non-linear Riemannian manifold of Symmetri…
Central South University（中南大学）；University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；Southwest Jiaotong University（西南交通大学）
Airborne Laser Scanning (ALS) point clouds have complex structures, and their 3D semantic labeling has been a challenging task. It has three problems: (1) the difficulty of classifying point clouds around boundaries of objects from different classes, (2) the diversity of shapes within the same class, and (3) the scale differences between classes. In this study, we propose a novel double self-attention convolutional network called the DAPnet. The double self-attention includes the point attention module (PAM) and the group attention module (GAM). For problem (1), the PAM can effectively assign …
The University of Texas at San Antonio（德克萨斯大学圣安东尼奥分校）
Global warming is an urgent issue that is generating catastrophic environmental changes, such as the melting of sea ice and glaciers, particularly in the polar regions. The melting pattern and retreat of polar sea ice cover is an essential indicator of global warming. The Sentinel-2 satellite (S2) captures high-resolution optical imagery over the polar regions. This research aims at developing a robust and effective system for classifying polar sea ice as thick or snow-covered, young or thin, or open water using S2 images. A key challenge is the lack of labeled S2 training data to serve as the…
Australian National University（澳大利亚国立大学）；Technical University Munich（慕尼黑工业大学）；Indian Institute of Technology Mandi（印度理工学院曼迪分校）
In this work, we address a challenging problem of fine-grained and coarse-grained recognition of object manipulation actions. Due to the variations in geometrical and motion constraints, there are different manipulations actions possible to perform different sets of actions with an object. Also, there are subtle movements involved to complete most of object manipulation actions. This makes the task of object manipulation action recognition difficult with only just the motion information. We propose to use grasp and motion-constraints information to recognise and understand action intention wit…
Newcastle University（纽卡斯尔大学）；Georgia Institute of Technology（佐治亚理工学院）
Sleep is a fundamental physiological process that is essential for sustaining a healthy body and mind. The gold standard for clinical sleep monitoring is polysomnography(PSG), based on which sleep can be categorized into five stages, including wake/rapid eye movement sleep (REM sleep)/Non-REM sleep 1 (N1)/Non-REM sleep 2 (N2)/Non-REM sleep 3 (N3). However, PSG is expensive, burdensome, and not suitable for daily use. For long-term sleep monitoring, ubiquitous sensing may be a solution. Most recently, cardiac and movement sensing has become popular in classifying three-stage sleep, since both m…
Despite recent advances in MOOC, the current e-learning systems have advantages of alleviating barriers by time differences, and geographically spatial separation between teachers and students. However, there has been a 'lack of supervision' problem that e-learner's learning unit state(LUS) can't be supervised automatically. In this paper, we present a fusion framework considering three channel data sources: 1) videos/images from a camera, 2) eye movement information tracked by a low solution eye tracker and 3) mouse movement. Based on these data modalities, we propose a novel approach of mult…
University of Peradeniya（佩拉德尼亚大学）；Western Sydney University（西悉尼大学）
Recognizing sEMG (Surface Electromyography) signals belonging to a particular action (e.g., lateral arm raise) automatically is a challenging task as EMG signals themselves have a lot of variation even for the same action due to several factors. To overcome this issue, there should be a proper separation which indicates similar patterns repetitively for a particular action in raw signals. A repetitive pattern is not always matched because the same action can be carried out with different time duration. Thus, a depth sensor (Kinect) was used for pattern identification where three joint angles w…
AI-Guided Feature Segmentation Techniques to Model Features from Single Crystal Diamond Growth
Fraunhofer USA, Center Mid-Atlantic（弗劳恩霍夫美国中大西洋中心）；Michigan State University（密歇根州立大学）；Fraunhofer USA, Center Midwest（弗劳恩霍夫美国中西部中心）；Department of Computer Science, University of Maryland（马里兰大学计算机科学系）
With the rapid commoditization of wearable sensors, detecting human movements from sensor datasets has become increasingly common over a wide range of applications. To detect activities, data scientists iteratively experiment with different classifiers before deciding which model to deploy. Effective reasoning about and comparison of alternative classifiers are crucial in successful model development. This is, however, inherently difficult in developing classifiers for sensor data, where the intricacy of long temporal sequences, high prediction frequency, and imprecise labeling make standard e…
Polytechnique Montréal（蒙特利尔高等理工学院）；The Montreal Institute for Learning Algorithms（蒙特利尔学习算法研究所）
We explore recurrent encoder multi-decoder neural network architectures for semi-supervised sequence classification and reconstruction. We find that the use of multiple reconstruction modules helps models generalize in a classification task when only a small amount of labeled data is available, which is often the case in practice. Such models provide useful high-level representations of motions allowing clustering, searching and faster labeling of new sequences. We also propose a new, realistic partitioning of a well-known, high quality motion-capture dataset for better evaluations. We further…
Bernoulli Institute/Department of Computer Science, University of Groningen（格罗宁根大学伯努利学院/计算机科学系）；University of Groningen（格罗宁根大学）；Bernoulli Institute（伯努利学院）；Department of Computer Science（计算机科学系）；Groningen（格罗宁根）
Sharpened dimensionality reduction (SDR), which belongs to the class of multidimensional projection techniques, has recently been introduced to tackle the challenges in the exploratory and visual analysis of high-dimensional data. SDR has been applied to various real-world datasets, such as human activity sensory data and astronomical datasets. However, manually labeling the samples from the generated projection are expensive. To address this problem, we propose here to use clustering methods such as k-means, Hierarchical Clustering, Density-Based Spatial Clustering of Applications with Noise …
The Visual Experience Dataset: Over 200 Recorded Hours of Integrated Eye Movement, Odometry, and Egocentric Video
Barnard College, Columbia University（哥伦比亚大学巴纳德学院）；Bates College（贝茨学院）；North Dakota State University（北达科他州立大学）；University of Nevada, Reno（内华达大学里诺分校）；Magic Leap；Technical University of Munich（慕尼黑工业大学）；Unmanned Ground Systems（无人地面系统公司）；Smith-Kettlewell Eye Research Institute（史密斯-凯特威尔眼科研究所）
Duke University（杜克大学）；Tsinghua University（清华大学）；Lund University（隆德大学）
Fast noninvasive probing of spatially varying decorrelating events, such as cerebral blood flow beneath the human skull, is an essential task in various scientific and clinical settings. One of the primary optical techniques used is diffuse correlation spectroscopy (DCS), whose classical implementation uses a single or few single-photon detectors, resulting in poor spatial localization accuracy and relatively low temporal resolution. Here, we propose a technique termed Classifying Rapid decorrelation Events via Parallelized single photon dEtection (CREPE)}, a new form of DCS that can probe and…
Graz University of Technology（格拉茨理工大学）；University of Graz（格拉茨大学）；Galileo University（伽利略大学）；University of Queensland（昆士兰大学）
Biomechanical and clinical gait research observes muscles and tendons in limbs to study their functions and behaviour. Therefore, movements of distinct anatomical landmarks, such as muscle-tendon junctions, are frequently measured. We propose a reliable and time efficient machine-learning approach to track these junctions in ultrasound videos and support clinical biomechanists in gait analysis. In order to facilitate this process, a method based on deep-learning was introduced. We gathered an extensive dataset, covering 3 functional movements, 2 muscles, collected on 123 healthy and 38 impaire…
