Soccer ball detection is identified as one of the critical challenges in the RoboCup competition. It requires an efficient vision system capable of handling the task of detection with high precision and recall and providing robust and low inference time. In this work, we present a novel convolutional neural network (CNN) approach to detect the soccer ball in an image sequence. In contrast to the existing methods where only the current frame or an image is used for the detection, we make use of the history of frames. Using history allows to efficiently track the ball in situations where the ball disappears or gets partially occluded in some of the frames. Our approach exploits spatio-temporal correlation and detects the ball based on the trajectory of its movements. We present our results with three convolutional methods, namely temporal convolutional networks (TCN), ConvLSTM, and ConvGRU. We first solve the detection task for an image using fully convolutional encoder-decoder architecture, and later, we use it as an input to our temporal models and jointly learn the detection task in sequences of images. We evaluate all our experiments on a novel dataset prepared as a part of this work. Furthermore, we present empirical results to support the effectiveness of using the history of the ball in challenging scenarios.
FEI University Center（FEI大学中心）；Federal Institute of Education, Science and Technology of São Paulo（圣保罗联邦教育、科学和技术学院）
Object detection techniques that achieve state-of-the-art detection accuracy employ convolutional neural networks, implemented to have optimal performance in graphics processing units. Some hardware systems, such as mobile robots, operate under constrained hardware situations, but still benefit from object detection capabilities. Multiple network models have been proposed, achieving comparable accuracy with reduced architectures and leaner operations. Motivated by the need to create an object detection system for a soccer team of mobile robots, this work provides a comparative study of recent …
Group activity detection in soccer can be done by using either video data or player and ball trajectory data. In current soccer activity datasets, activities are labelled as atomic events without a duration. Given that the state-of-the-art activity detection methods are not well-defined for atomic actions, these methods cannot be used. In this work, we evaluated the effectiveness of activity recognition models for detecting such events, by using an intuitive non-maximum suppression process and evaluation metrics. We also considered the problem of explicitly modeling interactions between player…
Qatar Computing Research Institute（卡塔尔计算研究所）；HBKU（哈马德·本·哈利法大学）；Telefonica Research（西班牙电信研究院）
Technology offers new ways to measure the locations of the players and of the ball in sports. This translates to the trajectories the ball takes on the field as a result of the tactics the team applies. The challenge professionals in soccer are facing is to take the reverse path: given the trajectories of the ball is it possible to infer the underlying strategy/tactic of a team? We propose a method based on Dynamic Time Warping to reveal the tactics of a team through the analysis of repeating series of events. Based on the analysis of an entire season, we derive insights such as passing strate…
Advanced Mining Technology Center（高级采矿技术中心）；Universidad de Chile（智利大学）
The goal of this paper is to propose a vision system for humanoid robotic soccer that does not use any color information. The main features of this system are: (i) real-time operation in the NAO robot, and (ii) the ability to detect the ball, the robots, their orientations, the lines and key field features robustly. Our ball detector, robot detector, and robot's orientation detector obtain the highest reported detection rates. The proposed vision system is tested in a SPL field with several NAO robots under realistic and highly demanding conditions. The obtained results are: robot detection ra…
The detection of small and medium-sized objects in three dimensions has always been a frontier exploration problem. This technology has a very wide application in sports analysis, games, virtual reality, human animation and other fields. The traditional three-dimensional small target detection technology has the disadvantages of high cost, low precision and inconvenience, so it is difficult to apply in practice. With the development of machine learning and deep learning, the technology of computer vision algorithms is becoming more mature. Creating an immersive media experience is considered t…
École Polytechnique Fédérale de Lausanne (EPFL)（洛桑联邦理工学院）
Tracking the ball is critical for video-based analysis of team sports. However, it is difficult, especially in low-resolution images, due to the small size of the ball, its speed that creates motion blur, and its often being occluded by players. In this paper, we propose a generic and principled approach to modeling the interaction between the ball and the players while also imposing appropriate physical constraints on the ball's trajectory. We show that our approach, formulated in terms of a Mixed Integer Program, is more robust and more accurate than several state-of-the-art approaches on re…
University of Liège（列日大学）；King Abdullah University of Science and Technology（阿卜杜拉国王科技大学）；Spiideo（斯皮迪奥公司）；Aalborg University（奥尔堡大学）；SpAItial（斯帕蒂亚尔公司）；Center for Robotics, Mines Paris, PSL（巴黎矿业学院机器人中心（巴黎文理研究大学））；Footovision（Footovision公司）；Shanghai Jiao Tong University（上海交通大学）；Universitat de Barcelona（巴塞罗那大学）；Computer Vision Center（计算机视觉中心）；EVS Broadcast Equipment（EVS广播设备公司）；Pioneer Center for Artificial Intelligence（先锋人工智能中心）；Lund University（隆德大学）；TAHAKOM（TAHAKOM公司）；Mohamed Bin Zayed University for Artificial Intelligence (MBZUAI)（穆罕默德·本·扎耶德人工智能大学）；Sber AI（Sber人工智能公司）；Salute For Business（Salute For Business公司）；Intelligent Perception and Image Understanding Lab, Xidian University（西安电子科技大学智能感知与图像理解实验室）；South China University of Technology（华南理工大学）；Hefei University of Technology（合肥工业大学）；Kyungpook National University（庆北国立大学）；Leipzig University of Applied Sciences（莱比锡应用科学大学）；Friedrich-Alexander University Erlangen-Nuremberg（埃尔朗根-纽伦堡大学）；University of Seoul（首尔大学）；Shenzhen Institute for Advanced Study, University of Electronic Science and Technology of China（电子科技大学深圳高等研究院）；Nanjing University（南京大学）；University of Science, Ho Chi Minh City（胡志明市科技大学）；Nanyang Technological University（南洋理工大学）；National University of Singapore（新加坡国立大学）
Augmented reality applications are beginning to change the way sports are broadcast, providing richer experiences and valuable insights to fans. The first step of augmented reality systems is camera calibration, possibly based on detecting the line markings of the playing field. Most existing proposals for line detection rely on edge detection and Hough transform, but radial distortion and extraneous edges cause inaccurate or spurious detections of line markings. We propose a novel strategy to automatically and accurately segment and classify line markings. First, line points are segmented tha…
University of Canterbury（坎特伯雷大学）；HIT Lab NZ（新西兰HIT实验室）
With the emergence of onboard vision processing for areas such as the internet of things (IoT), edge computing and autonomous robots, there is increasing demand for computationally efficient convolutional neural network (CNN) models to perform real-time object detection on resource constraints hardware devices. Tiny-YOLO is generally considered as one of the faster object detectors for low-end devices and is the basis for our work. Our experiments on this network have shown that Tiny-YOLO can achieve 0.14 frames per second(FPS) on the Raspberry Pi 3 B, which is too slow for soccer playing auto…
Warsaw University of Technology（华沙理工大学）；Sport Algorithmics and Gaming Sp. z o.o.（Sport Algorithmics and Gaming有限责任公司）
The paper describes a deep network based object detector specialized for ball detection in long shot videos. Due to its fully convolutional design, the method operates on images of any size and produces \emph{ball confidence map} encoding the position of detected ball. The network uses hypercolumn concept, where feature maps from different hierarchy levels of the deep convolutional network are combined and jointly fed to the convolutional classification layer. This allows boosting the detection accuracy as larger visual context around the object of interest is taken into account. The method ac…
University of Pisa（比萨大学）；ISTI-CNR（意大利国家研究委员会信息科学与技术研究所）
Soccer analytics is attracting increasing interest in academia and industry, thanks to the availability of data that describe all the spatio-temporal events that occur in each match. These events (e.g., passes, shots, fouls) are collected by human operators manually, constituting a considerable cost for data providers in terms of time and economic resources. In this paper, we describe PassNet, a method to recognize the most frequent events in soccer, i.e., passes, from video streams. Our model combines a set of artificial neural networks that perform feature extraction from video streams, obje…
Warsaw University of Technology（华沙理工大学）；Sport Algorithmics and Gaming Sp. z o.o.（体育算法与游戏股份有限公司）
The paper describes a deep neural network-based detector dedicated for ball and players detection in high resolution, long shot, video recordings of soccer matches. The detector, dubbed FootAndBall, has an efficient fully convolutional architecture and can operate on input video stream with an arbitrary resolution. It produces ball confidence map encoding the position of the detected ball, player confidence map and player bounding boxes tensor encoding players' positions and bounding boxes. The network uses Feature Pyramid Network desing pattern, where lower level features with higher spatial …
Centro de Informática, Universidade Federal de Pernambuco（伯南布哥联邦大学信息中心）
In the RoboCup Small Size League (SSL), teams are encouraged to propose solutions for executing basic soccer tasks inside the SSL field using only embedded sensing information. Thus, this work proposes an embedded monocular vision approach for detecting objects and estimating relative positions inside the soccer field. Prior knowledge from the environment is exploited by assuming objects lay on the ground, and the onboard camera has its position fixed on the robot. We implemented the proposed method on an NVIDIA Jetson Nano and employed SSD MobileNet v2 for 2D Object Detection with TensorRT op…
Qazvin Azad University（加兹温伊斯兰阿扎德大学）；University of Leeds（利兹大学）
In this paper, we present an active vision method using a deep reinforcement learning approach for a humanoid soccer-playing robot. The proposed method adaptively optimises the viewpoint of the robot to acquire the most useful landmarks for self-localisation while keeping the ball into its viewpoint. Active vision is critical for humanoid decision-maker robots with a limited field of view. To deal with an active vision problem, several probabilistic entropy-based approaches have previously been proposed which are highly dependent on the accuracy of the self-localisation model. However, in this…
In this technical report, we describe the use of a machine learning approach for detecting the realistic black and white ball currently in use in the RoboCup Standard Platform League. Our aim is to provide a ready-to-use software module that can be useful for the RoboCup SPL community. To this end, the approach is integrated within the official B-Human code release 2016. The complete code for the approach presented in this work can be downloaded from the SPQR Team homepage at http://spqr.diag.uniroma1.it and from the SPQR Team GitHub repository at https://github.com/SPQRTeam/SPQRBallPerceptor.…
