To date, machine learning for human action recognition in video has been widely implemented in sports activities. Although some studies have been successful in the past, precision is still the most significant concern. In this study, we present a high-accuracy framework to automatically clip the sports video stream by using a three-level prediction algorithm based on two classical open-source structures, i.e., YOLO-v3 and OpenPose. It is found that by using a modest amount of sports video training data, our methodology can perform sports activity highlights clipping accurately. Comparing with the previous systems, our methodology shows some advantages in accuracy. This study may serve as a new clipping system to extend the potential applications of the video summarization in sports field, as well as facilitates the development of match analysis system.
Human actions in videos are 3D signals. However, there are a few methods available for multiple human action recognition. For long videos, it's difficult to search within a video for a specific action and/or person. For that, this paper proposes a new technic for multiple human action recognition and summarization for surveillance videos. The proposed approach proposes a new representation of the data by extracting the sequence of each person from the scene. This is followed by an analysis of each sequence to detect and recognize the corresponding actions using 3D convolutional neural networks…
Video processing has become a popular research direction in computer vision due to its various applications such as video summarization, action recognition, etc. Recently, deep learning-based methods have achieved impressive results in action recognition. However, these methods need to process a full video sequence to recognize the action, even though most of these frames are similar and non-essential to recognizing a particular action. Additionally, these non-essential frames increase the computational cost and can confuse a method in action recognition. Instead, the important frames called k…
XRF V2: A Dataset for Action Summarization with Wi-Fi Signals, and IMUs in Phones, Watches, Earbuds, and Glasses
School of Software Engineering, Xi'an Jiaotong University（西安交通大学软件工程学院）；State Key Laboratory of Integrated Services Networks, Xidian University（西安电子科技大学信息网络服务国家重点实验室）；MOE KLINNS Lab, Xi'an Jiaotong University（教育部KLINNS实验室）；School of Computer Science and Technology, Xi'an Jiaotong University（西安交通大学计算机科学与技术学院）；College of Computer Science and Technology, Zhejiang University（浙江大学计算机科学与技术学院）
A Mobile Robot Generating Video Summaries of Seniors' Indoor Activities
National Taiwan University（台湾大学）；Intel Labs, Intel Technologies India Pvt. Ltd.（英特尔印度技术私人有限公司英特尔研究院）；Seoul National University（首尔大学）
我们开发了一个系统，该系统从社交机器人捕获的老年人室内活动视频中生成摘要，以帮助远程家庭成员了解他们的老年人在家中的日常活动。与传统的视频摘要数据集不同，从移动机器人捕获的室内视频提出了额外的挑战，即，（i）视频序列非常长（ii）大量的视频帧包含无主体或主体处于不适定位置和尺度（iii）大多数适定帧包含高度冗余信息。为了解决这个问题，我们建议\hl{exploit}构成估计\hl{for detecting}人在帧\hl{. This guides the robot}跟随用户和捕捉有效的视频。我们使用个人识别来区分目标老年人和其他人。本研究以\hl{also make use of}动作辨识分析老年人在不同时刻的主要活动，并发展出一套影像摘要方法，选取多样且具代表性的关键帧作为摘要。
Increasing volume of user-generated human-centric video content and their applications, such as video retrieval and browsing, require compact representations that are addressed by the video summarization literature. Current supervised studies formulate video summarization as a sequence-to-sequence learning problem and the existing solutions often neglect the surge of human-centric view, which inherently contains affective content. In this study, we investigate the affective-information enriched supervised video summarization task for human-centric videos. First, we train a visual input-driven …
In this paper we introduce OperA, a transformer-based model that accurately predicts surgical phases from long video sequences. A novel attention regularization loss encourages the model to focus on high-quality frames during training. Moreover, the attention weights are utilized to identify characteristic high attention frames for each surgical phase, which could further be used for surgery summarization. OperA is thoroughly evaluated on two datasets of laparoscopic cholecystectomy videos, outperforming various state-of-the-art temporal refinement approaches.
In this paper, we test the hypothesis that interesting events in unstructured videos are inherently audiovisual. We combine deep image representations for object recognition and scene understanding with representations from an audiovisual affect recognition model. To this set, we include content agnostic audio-visual synchrony representations and mel-frequency cepstral coefficients to capture other intrinsic properties of audio. These features are used in a modular supervised model. We present results from two experiments: efficacy study of single features on the task, and an ablation study wh…
MovieTeller: Tool-augmented Movie Synopsis with ID Consistent Progressive Abstraction
Central Universities（中央高校）；National Natural Science Foundation of China（中国国家自然科学基金委员会）；Research Fund for International Scientists of National Natural Science Foundation of China（中国国家自然科学基金委员会国际科学家研究基金）
