Action recognition is an important and challenging problem in video analysis. Although the past decade has witnessed progress in action recognition with the development of deep learning, such process has been slow in competitive sports content analysis. To promote the research on action recognition from competitive sports video clips, we introduce a Figure Skating Dataset (FSD-10) for finegrained sports content analysis. To this end, we collect 1484 clips from the worldwide figure skating championships in 2017-2018, which consist of 10 different actions in men/ladies programs. Each clip is at a rate of 30 frames per second with resolution 1080 $\times$ 720. These clips are then annotated by experts in type, grade of execution, skater info, .etc. To build a baseline for action recognition in figure skating, we evaluate state-of-the-art action recognition methods on FSD-10. Motivated by the idea that domain knowledge is of great concern in sports field, we propose a keyframe based temporal segment network (KTSN) for classification and achieve remarkable performance. Experimental results demonstrate that FSD-10 is an ideal dataset for benchmarking action recognition algorithms, as it requires to accurately extract action motions rather than action poses. We hope FSD-10, which is designed to have a large collection of finegrained actions, can serve as a new challenge to develop more robust and advanced action recognition models.
Institute of Microelectronics of Chinese Academy of Sciences（中国科学院微电子研究所）；University of Chinese Academy of Sciences（中国科学院大学）
To accurately and efficiently distinguish the stem end and the blossom end of navel orange from its black spots, we propose a feature skyscraper detector (FSD) with low computational cost, compact architecture and high detection accuracy. The main part of the detector is inspired from small object that stem (blossom) end is complex and black spot is densely distributed, so we design the feature skyscraper networks (FSN) based on dense connectivity. In particular, FSN is distinguished from regular feature pyramids, and which provides more intensive detection of high-level features. Then we desi…
SwimXYZ: A large-scale dataset of synthetic swimming motions and videos
CentraleSupélec（巴黎中央理工-高等电力学院）；IETR UMR CNRS 6164（法国国家科学研究中心雷恩电子与电信研究所（UMR 6164））；Centrale Nantes（南特中央理工学院）；Université Technologique de Compiègne（贡比涅技术大学）
FSD V2: Improving Fully Sparse 3D Object Detection with Virtual Voxels
Center for Research on Intelligent Perception and Computing (CRIPAC), National Laboratory of Pattern Recognition (NLPR), Institute of Automation, Chinese Academy of Sciences (CASIA)（中国科学院自动化研究所模式识别国家重点实验室智能感知与计算研究中心）；TuSimple（图森未来）
Cross-Block Fine-Grained Semantic Cascade for Skeleton-Based Sports Action Recognition
School of Automation, Southeast University（东南大学自动化学院）；School of Instrument Science and Engineering, Southeast University（东南大学仪器科学与工程学院）；Department of Computer and Information Science, University of Massachusetts Dartmouth（马萨诸塞大学达特茅斯分校计算机与信息科学系）
In problems such as sports video analytics, it is difficult to obtain accurate frame level annotations and exact event duration because of the lengthy videos and sheer volume of video data. This issue is even more pronounced in fast-paced sports such as ice hockey. Obtaining annotations on a coarse scale can be much more practical and time efficient. We propose the task of event detection in coarsely annotated videos. We introduce a multi-tower temporal convolutional network architecture for the proposed task. The network, with the help of multiple receptive fields, processes information at va…
Tsinghua University（清华大学）；Beijing National Research Center for Information Science and Technology（北京信息科学与技术国家研究中心）
Most existing action quality assessment methods rely on the deep features of an entire video to predict the score, which is less reliable due to the non-transparent inference process and poor interpretability. We argue that understanding both high-level semantics and internal temporal structures of actions in competitive sports videos is the key to making predictions accurate and interpretable. Towards this goal, we construct a new fine-grained dataset, called FineDiving, developed on diverse diving events with detailed annotations on action procedures. We also propose a procedure-aware approa…
Indian Institute of Technology Gandhinagar（甘地讷格尔印度理工学院）
Competitive diving is a well recognized aquatic sport in which a person dives from a platform or a springboard into the water. Based on the acrobatics performed during the dive, diving is classified into a finite set of action classes which are standardized by FINA. In this work, we propose an attention guided LSTM-based neural network architecture for the task of diving classification. The network takes the frames of a diving video as input and determines its class. We evaluate the performance of the proposed model on a recently introduced competitive diving dataset, Diving48. It contains ove…
University of the Bundeswehr Munich（慕尼黑联邦国防军大学）；Institute for Distributed Intelligent Systems（分布式智能系统研究所）
Sports analytics benefits from recent advances in machine learning providing a competitive advantage for teams or individuals. One important task in this context is the performance measurement of individual players to provide reports and log files for subsequent analysis. During sport events like basketball, this involves the re-identification of players during a match either from multiple camera viewpoints or from a single camera viewpoint at different times. In this work, we investigate whether it is possible to transfer the out-standing zero-shot performance of pre-trained CLIP models to th…
FLEX: A Largescale Multimodal, Multiview Dataset for Learning Structured Representations for Fitness Action Quality Assessment
School of Biomedical Engineering (Suzhou), USTC（中国科学技术大学苏州生物医学工程学院）；Suzhou Institute of Biomedical Engineering and Technology, CAS（中国科学院苏州生物医学工程技术研究所）；Institute of High-Performance Computing, A*STAR, Singapore（新加坡科技研究局高性能计算研究所）；School of Psychology, Beijing Sports University（北京体育大学心理学院）；School of Competitive Sports, Beijing Sports University（北京体育大学竞技体育学院）；Department of Robotics, University of Michigan（密歇根大学机器人系）
Motion is a salient cue to recognize actions in video. Modern action recognition models leverage motion information either explicitly by using optical flow as input or implicitly by means of 3D convolutional filters that simultaneously capture appearance and motion information. This paper proposes an alternative approach based on a learnable correlation operator that can be used to establish frame-toframe matches over convolutional feature maps in the different layers of the network. The proposed architecture enables the fusion of this explicit temporal matching information with traditional ap…
SoccerChat: Integrating Multimodal Data for Enhanced Soccer Game Understanding
SimulaMet and OsloMet, Norway（SimulaMet和OsloMet，挪威）；Forzasys, Norway（Forzasys，挪威）；SimulaMet, Norway（SimulaMet，挪威）；SimulaMet, OsloMet, and Forzasys, Norway（SimulaMet、OsloMet和Forzasys，挪威）；University of Central Florida, USA（佛罗里达中央大学）
University of Maryland, College Park（马里兰大学帕克分校）；Allen Institute for AI（艾伦人工智能研究所）；Comcast Labs, DC（康卡斯特实验室（华盛顿特区））
Even with the recent advances in convolutional neural networks (CNN) in various visual recognition tasks, the state-of-the-art action recognition system still relies on hand crafted motion feature such as optical flow to achieve the best performance. We propose a multitask learning model ActionFlowNet to train a single stream network directly from raw pixels to jointly estimate optical flow while recognizing actions with convolutional neural networks, capturing both appearance and motion in a single model. We additionally provide insights to how the quality of the learned optical flow affects …
University of Amsterdam（阿姆斯特丹大学）；Delft University of Technology（代尔夫特理工大学）
We strive for spatio-temporal localization of actions in videos. The state-of-the-art relies on action proposals at test time and selects the best one with a classifier trained on carefully annotated box annotations. Annotating action boxes in video is cumbersome, tedious, and error prone. Rather than annotating boxes, we propose to annotate actions in video with points on a sparse subset of frames only. We introduce an overlap measure between action proposals and points and incorporate them all into the objective of a non-convex Multiple Instance Learning optimization. Experimental evaluation…
Oklahoma State University（俄克拉荷马州立大学）；University of South Florida（南佛罗里达大学）
The problem of action recognition involves locating the action in the video, both over time and spatially in the image. The dominant current approaches use supervised learning to solve this problem, and require large amounts of annotated training data, in the form of frame-level bounding box annotations around the region of interest. In this paper, we present a new approach based on continual learning that uses feature-level predictions for self-supervision. It does not require any training annotations in terms of frame-level bounding boxes. The approach is inspired by cognitive models of visu…
