Acquiring the accurate 3-D position of a target person around a robot provides fundamental and valuable information that is applicable to a wide range of robotic tasks, including home service, navigation and entertainment. This paper presents a real-time robotic 3-D human tracking system which combines a monocular camera with an ultrasonic sensor by the extended Kalman filter (EKF). The proposed system consists of three sub-modules: monocular camera sensor tracking model, ultrasonic sensor tracking model and multi-sensor fusion. An improved visual tracking algorithm is presented to provide partial location estimation (2-D). The algorithm is designed to overcome severe occlusions, scale variation, target missing and achieve robust re-detection. The scale accuracy is further enhanced by the estimated 3-D information. An ultrasonic sensor array is employed to provide the range information from the target person to the robot and Gaussian Process Regression is used for partial location estimation (2-D). EKF is adopted to sequentially process multiple, heterogeneous measurements arriving in an asynchronous order from the vision sensor and the ultrasonic sensor separately. In the experiments, the proposed tracking system is tested in both simulation platform and actual mobile robot for various indoor and outdoor scenes. The experimental results show the superior performance of the 3-D tracking system in terms of both the accuracy and robustness.
LE2I-CNRS, University of Technology of Belfort-Montbéliard（法国国家科学研究中心LE2I实验室，贝尔福-蒙贝利亚尔技术大学）；University of Lincoln（林肯大学）
Human detection and tracking is an essential task for service robots, where the combined use of multiple sensors has potential advantages that are yet to be exploited. In this paper, we introduce a framework allowing a robot to learn a new 3D LiDAR-based human classifier from other sensors over time, taking advantage of a multisensor tracking system. The main innovation is the use of different detectors for existing sensors (i.e. RGB-D camera, 2D LiDAR) to train, online, a new 3D LiDAR-based human classifier, exploiting a so-called trajectory probability. Our framework uses this probability to…
The field of robotic vision is developing rapidly. Robots can react intelligently and provide assistance to user activities through sentient computing. Since industrial applications pose complex requirements that cannot be handled by humans, an efficient low cost and robust technique is required for the tracking of mobile industrial robots. The existing sensor based techniques for mobile robot tracking are expensive and complex to deploy, configure and maintain. Also some of them demand dedicated and often expensive hardware. This paper presents a low cost vision based technique called Hybrid …
This paper proposes a process that uses two cameras to obtain three-dimensional (3D) information of a target object for human tracking. Results of human detection and tracking from two cameras are integrated to obtain the 3D information. OpenPose is used for human detection. In the case of a general processing a stereo camera, a range image of the entire scene is acquired as precisely as possible, and then the range image is processed. However, there are problems such as incorrect matching and computational cost for the calibration process. A new stereo vision framework is proposed to cope wit…
In this paper, a novel dual-mode robust model predictive control (MPC) approach is proposed for solving the tracking control problem of non-holonomoic mobile robots with additive bounded disturbance. To reduce the negative effect of disturbance and drive the state of real system closer to the one of nominal system , a robust reference signal is introduced into the cost function of MPC. In order to reduced the computation burden caused by online optimization of MPC and further improve the tracking accuracy, a dual-mode control strucuture consisting of the robust MPC and the local nonlinear robu…
University of Zagreb Faculty of Electrical Engineering and Computing（萨格勒布大学电气工程与计算学院）
Perception is one of the key abilities of autonomous mobile robotic systems, which often relies on fusion of heterogeneous sensors. Although this heterogeneity presents a challenge for sensor calibration, it is also the main prospect for reliability and robustness of autonomous systems. In this paper, we propose a method for multisensor calibration based on Gaussian processes (GPs) estimated moving object trajectories, resulting with temporal and extrinsic parameters. The appealing properties of the proposed temporal calibration method are: coordinate frame invariance, thus avoiding prior extr…
National Cheng Kung University（成功大学）；University of Hawai‘i at Mānoa（夏威夷大学马诺阿分校）
This paper investigates the resilient control, analysis, recovery, and operation of mobile robot networks in time-varying formation tracking under deception attacks on global positioning. Local and global tracking control algorithms are presented to ensure redundancy of the mobile robot network and to retain the desired functionality for better resilience. Lyapunov stability analysis is utilized to show the boundedness of the formation tracking error and the stability of the network under various attack modes. A performance index is designed to compare the efficiency of the proposed formation …
In this paper, we construct a lightweight, high-precision and high-speed object tracking using a trained CNN. Conventional methods with trained CNNs use VGG16 network which requires powerful computational resources. Therefore, there is a problem that it is difficult to apply in low computation resources environments. To solve this problem, we use MobileNetV3, which is a CNN for mobile terminals.Based on Feature Map Selection Tracking, we propose a new architecture that extracts effective features of MobileNet for object tracking. The architecture requires no online learning but only offline le…
Massachusetts Institute of Technology（麻省理工学院）；University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）
This paper presents a Tracking-Error Learning Control (TELC) algorithm for precise mobile robot path tracking in off-road terrain. In traditional tracking error-based control approaches, feedback and feedforward controllers are designed based on the nominal model which cannot capture the uncertainties, disturbances and changing working conditions so that they cannot ensure precise path tracking performance in the outdoor environment. In TELC algorithm, the feedforward control actions are updated by using the tracking error dynamics and the plant-model mismatch problem is thus discarded. Theref…
INRIA（法国国家信息与自动化研究所）；Phu Xuan Private University（富春私立大学）
Mobile object tracking has an important role in the computer vision applications. In this paper, we use a tracked target-based taxonomy to present the object tracking algorithms. The tracked targets are divided into three categories: points of interest, appearance and silhouette of mobile objects. Advantages and limitations of the tracking approaches are also analyzed to find the future directions in the object tracking domain.
ROS 2-Based LiDAR Perception Framework for Mobile Robots in Dynamic Production Environments, Utilizing Synthetic Data Generation, Transformation-Equivariant 3D Detection and Multi-Object Tracking
Chair of Intelligence in Quality Sensing (IQS), Laboratory for Machine Tools and Production Engineering (WZL), RWTH Aachen University（亚琛工业大学机床与生产工程实验室质量感知智能教席）；The Sirindhorn International Thai-German Graduate School of Engineering (TGGS), King Mongkut’s University of Technology North Bangkok (KMUTNB)（北曼谷先皇技术大学泰德国际工程研究生院）
Tracking object poses in 3D is a crucial building block for Augmented Reality applications. We propose an instant motion tracking system that tracks an object's pose in space (represented by its 3D bounding box) in real-time on mobile devices. Our system does not require any prior sensory calibration or initialization to function. We employ a deep neural network to detect objects and estimate their initial 3D pose. Then the estimated pose is tracked using a robust planar tracker. Our tracker is capable of performing relative-scale 9-DoF tracking in real-time on mobile devices. By combining use…
This paper presents a new algorithm to track mobile objects in different scene conditions. The main idea of the proposed tracker includes estimation, multi-features similarity measures and trajectory filtering. A feature set (distance, area, shape ratio, color histogram) is defined for each tracked object to search for the best matching object. Its best matching object and its state estimated by the Kalman filter are combined to update position and size of the tracked object. However, the mobile object trajectories are usually fragmented because of occlusions and misdetections. Therefore, we a…
Slip is a very common phenomena present in wheeled mobile robotic systems. It has undesirable consequences such as wasting energy and impeding system stability. To tackle the challenge of mobile robot trajectory tracking under slippery conditions, we propose a hierarchical framework that learns and adapts gains of the tracking controllers simultaneously online. Concretely, a reinforcement learning (RL) module is used to auto-tune parameters in a lateral predictive controller and a longitudinal speed PID controller. Experiments show the necessity of simultaneous gain tuning, and have demonstrat…
Recognizing a previously visited place, also known as place recognition (or loop closure detection) is the key towards fully autonomous mobile robots and self-driving vehicle navigation. Augmented with various Simultaneous Localization and Mapping techniques (SLAM), loop closure detection allows for incremental pose correction and can bolster efficient and accurate map creation. However, repeated and similar scenes (perceptual aliasing) and long term appearance changes (e.g. weather variations) are major challenges for current place recognition algorithms. We introduce a new dataset Multisenso…
In the last decades, visual target tracking has been one of the primary research interests of the Robotics research community. The recent advances in Deep Learning technologies have made the exploitation of visual tracking approaches effective and possible in a wide variety of applications, ranging from automotive to surveillance and human assistance. However, the majority of the existing works focus exclusively on passive visual tracking, i.e., tracking elements in sequences of images by assuming that no actions can be taken to adapt the camera position to the motion of the tracked entity. On…
In this paper we present The Rosario Dataset, a collection of sensor data for autonomous mobile robotics in agricultural scenes. The dataset is motivated by the lack of realistic sensor readings gathered by a mobile robot in such environments. It consists of 6 sequences recorded in soybean fields showing real and challenging cases: highly repetitive scenes, reflection and burned images caused by direct sunlight and rough terrain among others. The dataset was conceived in order to provide a benchmark and contribute to the agricultural SLAM/odometry and sensor fusion research. It contains synchr…
In the current era of the industrial revolution, mobile robots are playing a pivotal role in helping out mankind in many complex and hazardous environments for performing tasks like search and rescue, obstacle avoidance, mining and security surveillance, etc. A lot of navigation algorithms have been developed in recent years but novel challenges still exist in autonomous path planning of multiple robots to track and follow multiple intruders. This report demonstrates a decentralized strategy of arithmetic mean based navigation algorithm for a group of mobile robots to navigate through an unkno…
ERPoT: Effective and Reliable Pose Tracking for Mobile Robots Using Lightweight Polygon Maps
ZJU-Hangzhou Global Scientific and Technological Innovation Center, Zhejiang University（浙江大学杭州全球科技创新中心）；State Key Lab of CAD&CG, Zhejiang University（浙江大学计算机辅助设计与图形学国家重点实验室）；China Mobile (Zhejiang) Research & Innovation Institute（中国移动（浙江）研究院）；Beijing Fuyouhua Intelligent Technology Co., Ltd.（北京傅友华智能科技有限公司）；College of Information Engineering, Zhejiang University of Technology（浙江工业大学信息工程学院）；School of Control Science and Engineering, Shandong University（山东大学控制科学与工程学院）；Institute of Robotics and Automatic Information System (IRAIS), Tianjin Key Laboratory of Intelligent Robotics (TJKLIR), Nankai University（机器人与自动信息系统研究院，天津智能机器人重点实验室，南开大学）
CSIRO ICT Centre（澳大利亚联邦科学与工业研究组织信息与通信技术中心）；Université de Sherbrooke（舍布鲁克大学）
Mobile robots in real-life settings would benefit from being able to localize and track sound sources. Such a capability can help localizing a person or an interesting event in the environment, and also provides enhanced processing for other capabilities such as speech recognition. To give this capability to a robot, the challenge is not only to localize simultaneous sound sources, but to track them over time. In this paper we propose a robust sound source localization and tracking method using an array of eight microphones. The method is based on a frequency-domain implementation of a steered…
Multimodal Anomaly Detection based on Deep Auto-Encoder for Object Slip Perception of Mobile Manipulation Robots
Artificial Intelligence Institute, Seoul National University（首尔国立大学人工智能研究所）；Dept. of Computer Science and Engineering, Seoul National University（首尔国立大学计算机科学与工程系）；Interdisciplinary Program in Neuroscience, Seoul National University（首尔国立大学神经科学跨学科项目）；Surromind Inc.（Surromind公司）
- Both Lidars and Radars are sensors for obstacle detection. While Lidars are very accurate on obstacles positions and less accurate on their velocities, Radars are more precise on obstacles velocities and less precise on their positions. Sensor fusion between Lidar and Radar aims at improving obstacle detection using advantages of the two sensors. The present paper proposes a real-time Lidar/Radar data fusion algorithm for obstacle detection and tracking based on the global nearest neighbour standard filter (GNN). This algorithm is implemented and embedded in an automative vehicle as a compon…
Real-Time Nonlinear Model Predictive Control of Heavy-Duty Skid-Steered Mobile Platform for Trajectory Tracking Tasks
Unit of Automation Technology and Mechanical Engineering, Faculty of Engineering and Natural Sciences, Tampere University（自动化技术与机械工程单元，工程与自然科学学院，塔尔皮亚大学）
Virginia Tech（弗吉尼亚理工大学）；Massachusetts Institute of Technology（麻省理工学院）；University of Maryland（马里兰大学）
We study the problem of searching for and tracking a collection of moving targets using a robot with a limited Field-Of-View (FOV) sensor. The actual number of targets present in the environment is not known a priori. We propose a search and tracking framework based on the concept of Bayesian Random Finite Sets (RFSs). Specifically, we generalize the Gaussian Mixture Probability Hypothesis Density (GM-PHD) filter which was previously applied for tracking problems to allow for simultaneous search and tracking with a limited FOV sensor. The proposed framework can extract individual target tracks…
Shenyang Institute of Automation, Chinese Academy of Sciences（中国科学院沈阳自动化研究所）；Institutes for Robotics and Intelligent Manufacturing, Chinese Academy of Sciences（中国科学院机器人与智能制造创新研究院）；University of Chinese Academy of Sciences（中国科学院大学）；Heriot-Watt University（赫瑞-瓦特大学）
Agile control of mobile manipulator is challenging because of the high complexity coupled by the robotic system and the unstructured working environment. Tracking and grasping a dynamic object with a random trajectory is even harder. In this paper, a multi-task reinforcement learning-based mobile manipulation control framework is proposed to achieve general dynamic object tracking and grasping. Several basic types of dynamic trajectories are chosen as the task training set. To improve the policy generalization in practice, random noise and dynamics randomization are introduced during the train…
Harbin Institute of Technology Shenzhen（哈尔滨工业大学深圳校区）
To promote the widespread use of mobile robots in diverse fields, the performance of trajectory tracking must be ensured. To address the constraints and nonlinear features associated with mobile robot systems, we apply nonlinear model predictive control (MPC) to realize the trajectory tracking of mobile robots. Specifically, to alleviate the online computational complexity of nonlinear MPC, this paper devises a lattice piecewise affine (PWA) approximation method that can approximate both the nonlinear system and control law of explicit nonlinear MPC. The kinematic model of the mobile robot is …
The University of Adelaide（阿德莱德大学）；Curtin University（科廷大学）
We consider the challenging problem of online planning for a team of agents to autonomously search and track a time-varying number of mobile objects under the practical constraint of detection range limited onboard sensors. A standard POMDP with a value function that either encourages discovery or accurate tracking of mobile objects is inadequate to simultaneously meet the conflicting goals of searching for undiscovered mobile objects whilst keeping track of discovered objects. The planning problem is further complicated by misdetections or false detections of objects caused by range limited s…
We present Autonomous Rssi based RElative poSitioning and Tracking (ARREST), a new robotic sensing system for tracking and following a moving, RF-emitting object, which we refer to as the Leader, solely based on signal strength information. This kind of system can expand the horizon of autonomous mobile tracking and distributed robotics into many scenarios with limited visibility such as nighttime, dense forests, and cluttered environments. Our proposed tracking agent, which we refer to as the TrackBot, uses a single rotating, off-the-shelf, directional antenna, novel angle and relative speed …
