Predicting the future behavior of moving agents is essential for real world applications. It is challenging as the intent of the agent and the corresponding behavior is unknown and intrinsically multimodal. Our key insight is that for prediction within a moderate time horizon, the future modes can be effectively captured by a set of target states. This leads to our target-driven trajectory prediction (TNT) framework. TNT has three stages which are trained end-to-end. It first predicts an agent's potential target states $T$ steps into the future, by encoding its interactions with the environment and the other agents. TNT then generates trajectory state sequences conditioned on targets. A final stage estimates trajectory likelihoods and a final compact set of trajectory predictions is selected. This is in contrast to previous work which models agent intents as latent variables, and relies on test-time sampling to generate diverse trajectories. We benchmark TNT on trajectory prediction of vehicles and pedestrians, where we outperform state-of-the-art on Argoverse Forecasting, INTERACTION, Stanford Drone and an in-house Pedestrian-at-Intersection dataset.
The Chinese University of Hong Kong（香港中文大学）；Southern University of Science and Technology（南方科技大学）；Shenzhen Research Institute of the Chinese University of Hong Kong（香港中文大学深圳研究院）
With the rapid development of machine learning, autonomous driving has become a hot issue, making urgent demands for more intelligent perception and planning systems. Self-driving cars can avoid traffic crashes with precisely predicted future trajectories of surrounding vehicles. In this work, we review and categorize existing learning-based trajectory forecasting methods from perspectives of representation, modeling, and learning. Moreover, we make our implementation of Target-driveN Trajectory Prediction publicly available at https://github.com/Henry1iu/TNT-Trajectory-Predition, demonstratin…
Jet Propulsion Laboratory, California Institute of Technology（加州理工学院喷气推进实验室）
We propose TRADE for robust tracking and 3D localization of a moving target in cluttered environments, from UAVs equipped with a single camera. Ultimately TRADE enables 3d-aware target following. Tracking-by-detection approaches are vulnerable to target switching, especially between similar objects. Thus, TRADE predicts and incorporates the target 3D trajectory to select the right target from the tracker's response map. Unlike static environments, depth estimation of a moving target from a single camera is a ill-posed problem. Therefore we propose a novel 3D localization method for ground targ…
This work investigates an efficient trajectory generation for chasing a dynamic target, which incorporates the detectability objective. The proposed method actively guides the motion of a cinematographer drone so that the color of a target is well-distinguished against the colors of the background in the view of the drone. For the objective, we define a measure of color detectability given a chasing path. After computing a discrete path optimized for the metric, we generate a dynamically feasible trajectory. The whole pipeline can be updated on-the-fly to respond to the motion of the target. F…
This paper proposes a systematic solution that uses an unmanned aerial vehicle (UAV) to aggressively and safely track an agile target. The solution properly handles the challenging situations where the intent of the target and the dense environments are unknown to the UAV. Our work is divided into two parts: target motion prediction and tracking trajectory planning. The target motion prediction method utilizes target observations to reliably predict the future motion of the target considering dynamic constraints. The tracking trajectory planner follows the traditional hierarchical workflow.A t…
In this paper, we present a motion planning strategy for UAVs that generates a time-optimal trajectory to survey a given target area. There are several situations where completing an aerial survey is time sensitive, such as gaining situational awareness for first responders, surveying hazardous environments, etc. One of the challenges in such cases is to plan a time-optimal trajectory for the drone. To this end, we present an autonomous aerial survey framework that minimizes the time taken to completely explore a target area or volume using drones. In this work, (i) we present an approach, whe…
Homotopic information gain for sparse active target tracking
Australian Centre for Robotics, School of Aerospace, Mechanical and Mechatronic Engineering, University of Sydney（澳大利亚机器人中心，航空航天、机械与机电工程学院，悉尼大学）；Department of Electrical and Computer Engineering, UC San Diego（电气与计算机工程系，UC圣地亚哥大学）；School of Mechanical and Mechatronics Engineering, University of Technology Sydney（机械与机电工程学院，技术大学悉尼）
We studied a target tracking algorithm based on millimeter-wave (MMW) radar in an autonomous driving environment. Aiming at the cluster matching in the target tracking stage, a new weighted feature similarity algorithm is proposed, which increases the matching rate of the same target in adjacent frames under strong environmental noise and multiple interference targets. For autonomous driving scenarios, we constructed a method that uses its motion parameters to extract and correct the trajectory of a moving target, which solves the problem of moving target detection and trajectory correction du…
Faculty of Electrical Engineering and Computing, University of Zagreb（萨格勒布大学电气工程与计算学院）
This paper presents our approach to intercepting a faster intruder UAV, inspired by the MBZIRC 2020 Challenge 1. By utilizing a priori knowledge of the shape of the intruder's trajectory, we can calculate an interception point. Target tracking is based on image processing by a YOLOv3 Tiny convolutional neural network, combined with depth calculation using a gimbal-mounted ZED Mini stereo camera. We use RGB and depth data from the camera, devising a noise-reducing histogram-filter to extract the target's 3D position. Obtained 3D measurements of target's position are used to calculate the positi…
The interception of moving targets is a widely studied issue. In this paper, we propose an algorithm of intercepting the moving target with a wheeled mobile robot in a dynamic environment. We first predict the future position of the target through polynomial fitting. The algorithm then generates an interception trajectory with path and speed decoupling. We use Hybrid A* search to plan a path and optimize it via gradient decent method. To avoid the dynamic obstacles in the environment, we introduce ST graph for speed planning. The speed curve is represented by piecewise Bézier curves for furthe…
Collaborative target-tracking control using multiple autonomous fixed-wing UAVs with constant speeds
Department of Electrical Engineering, Eindhoven University of Technology（埃因霍温理工大学电子工程系）；Universidad Complutense de Madrid（马德里complutense大学）；Research School of Electrical, Energy and Material Engineering, Australian National University（澳大利亚国立大学电气、能源和材料工程研究学校）；Optus-Curtin Centre of Excellence in Artificial Intelligence, Curtin University（Curtin大学人工智能卓越中心）
Trajectory Optimization in Single and Dual-UAV Bearing-Only Target Localization
College of Aerospace Science and Engineering, National University of Defense Technology（国防科技大学航天科学与工程学院）；Hunan Key Laboratory of Image Measurement and Visual Navigation（湖南省图像测量与视觉导航重点实验室）
In Argoverse motion forecasting competition, the task is to predict the probabilistic future trajectory distribution for the interested targets in the traffic scene. We use vectorized lane map and 2 s targets' history trajectories as input. Then the model outputs 6 forecasted trajectories with probability for each target.
Dim Small Target Detection and Tracking: A Novel Method Based on Temporal Energy Selective Scaling and Trajectory Association
National Space Science Center, Chinese Academy of Sciences（中国科学院国家空间科学中心）；School of Computer Science and Technology, University of Chinese Academy of Sciences（中国科学院大学计算机科学与技术学院）
University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；University of Michigan（密歇根大学）
Multi-pedestrian trajectory prediction is an indispensable element of autonomous systems that safely interact with crowds in unstructured environments. Many recent efforts in trajectory prediction algorithms have focused on understanding social norms behind pedestrian motions. Yet we observe these works usually hold two assumptions, which prevent them from being smoothly applied to robot applications: (1) positions of all pedestrians are consistently tracked, and (2) the target agent pays attention to all pedestrians in the scene. The first assumption leads to biased interaction modeling with …
3D Trajectory Reconstruction of Moving Points Based on Asynchronous Cameras
College of Aerospace Science and Engineering, National University of Defense Technology（航空科学与工程学院，国防科技大学）；Hunan Provincial Key Laboratory of Image Measurement and Vision Navigation（湖南省图像测量与视觉导航重点实验室）
Huazhong Univerisity of Science and Technology（华中科技大学）；Tsinghua University（清华大学）
Discriminative representation is crucial for the association step in multi-object tracking. Recent work mainly utilizes features in single or neighboring frames for constructing metric loss and empowering networks to extract representation of targets. Although this strategy is effective, it fails to fully exploit the information contained in a whole trajectory. To this end, we propose a strategy, namely multi-view trajectory contrastive learning, in which each trajectory is represented as a center vector. By maintaining all the vectors in a dynamically updated memory bank, a trajectory-level c…
Anhui University（安徽大学）；The University of Sydney（悉尼大学）；Peng Cheng Laboratory（鹏城实验室）；Peking University（北京大学）；University of Science and Technology of China（中国科学技术大学）
Most of the existing single object trackers track the target in a unitary local search window, making them particularly vulnerable to challenging factors such as heavy occlusions and out-of-view movements. Despite the attempts to further incorporate global search, prevailing mechanisms that cooperate local and global search are relatively static, thus are still sub-optimal for improving tracking performance. By further studying the local and global search results, we raise a question: can we allow more dynamics for cooperating both results? In this paper, we propose to introduce more dynamics …
Nanjing University of Science and Technology（南京理工大学）；Yale-NUS College（耶鲁-新加坡国立大学学院）；National University of Singapore（新加坡国立大学）
Occlusion is a long-standing problem that causes many modern tracking methods to be erroneous. In this paper, we address the occlusion problem by exploiting the current and future possible locations of the target object from its past trajectory. To achieve this, we introduce a learning-based tracking method that takes into account background motion modeling and trajectory prediction. Our trajectory prediction module predicts the target object's locations in the current and future frames based on the object's past trajectory. Since, in the input video, the target object's trajectory is not only…
University of Washington（华盛顿大学）；Electronics and Telecommunications Research Institute（电子通信研究院）
In this paper, we propose a novel framework for multi-target multi-camera tracking (MTMCT) of vehicles based on metadata-aided re-identification (MA-ReID) and the trajectory-based camera link model (TCLM). Given a video sequence and the corresponding frame-by-frame vehicle detections, we first address the isolated tracklets issue from single camera tracking (SCT) by the proposed traffic-aware single-camera tracking (TSCT). Then, after automatically constructing the TCLM, we solve MTMCT by the MA-ReID. The TCLM is generated from camera topological configuration to obtain the spatial and tempora…
University of Michigan, Ann Arbor（密歇根大学安娜堡分校）；Fudan University（复旦大学）；ShanghaiTech University（上海科技大学）
Most pedestrian trajectory prediction methods rely on a huge amount of trajectories annotation, which is time-consuming and expensive. Moreover, a well-trained model may not effectively generalize to a new scenario captured by another camera. Therefore, it is desirable to adapt the model trained on an annotated source domain to the target domain. To achieve domain adaptation for trajectory prediction, we propose a Cross-domain Trajectory Prediction Network (CTP-Net). In this framework, encoders are used in both domains to encode the observed trajectories, then their features are aligned by a c…
The trajectory prediction is significant for the decision-making of autonomous driving vehicles. In this paper, we propose a model to predict the trajectories of target agents around an autonomous vehicle. The main idea of our method is considering the history trajectories of the target agent and the influence of surrounding agents on the target agent. To this end, we encode the target agent history trajectories as an attention mask and construct a social map to encode the interactive relationship between the target agent and its surrounding agents. Given a trajectory sequence, the LSTM networ…
Hefei University of Technology（合肥工业大学）；Jianghuai Advance Technology Center（江淮先进技术中心）；Anhui Provincial Key Laboratory of Humanoid Robots（安徽省人形机器人重点实验室）；Kaiyang Laboratory, Chery（凯阳实验室，中国奇瑞）；Zhejiang University of Technology（浙江工业大学）
ReferFlow: Beyond Final-Target Localization for Complex Referring Expressions
State Key Laboratory of Human-Machine Hybrid Augmented Intelligence, Institute of Artificial Intelligence and Robotics, Xi’an Jiaotong University（西安交通大学人工智能与机器人研究所人机混合增强智能全国重点实验室）；Xingchen AGI Lab, China Telecom Artificial Intelligence Technology (Beijing) Co., Ltd.（中国电信人工智能技术（北京）有限公司星辰AGI实验室）；Beijing Normal University（北京师范大学）；Zhejiang University（浙江大学）；University of the Chinese Academy of Sciences（中国科学院大学）；University of Science and Technology Beijing（北京科技大学）；Shanghai Jiao Tong University（上海交通大学）
Predicting the possible future trajectories of the surrounding dynamic agents is an essential requirement in autonomous driving. These trajectories mainly depend on the surrounding static environment, as well as the past movements of those dynamic agents. Furthermore, the multimodal nature of agent intentions makes the trajectory prediction problem more challenging. All of the existing models consider the target agent as well as the surrounding agents similarly, without considering the variation of physical properties. In this paper, we present a novel deep-learning based framework for multimo…
Poisson multi-Bernoulli mixture filter for trajectory measurements
Department of Electrical Engineering and Electronics, University of Liverpool（利兹大学电子工程与电子系）；IPTC, ETSI de Telecomunicación, Universidad Politécnica de Madrid（马德里理工大学电信工程系）
Inferring relational behavior between road users as well as road users and their surrounding physical space is an important step toward effective modeling and prediction of navigation strategies adopted by participants in road scenes. To this end, we propose a relation-aware framework for future trajectory forecast. Our system aims to infer relational information from the interactions of road users with each other and with the environment. The first module involves visual encoding of spatio-temporal features, which captures human-human and human-space interactions over time. The following modu…
Uncovering the human motion pattern: Pattern Memory-based Diffusion Model for Trajectory Prediction
Beijing Key Laboratory of Intelligent Telecommunications Software and Multimedia（智能电信软件与多媒体北京市重点实验室）；Beijing University of Posts and Telecommunications（北京邮电大学）
VisualErase: Dual-Branch Visual Trajectory Redirection for Robust Concept Erasure in Text-to-Image Diffusion Models
Harbin Institute of Technology (Shenzhen)（哈尔滨工业大学（深圳））；City University of Hong Kong（香港城市大学）；Shenzhen Loop Area Institute（深圳河套学院）；National University of Singapore（新加坡国立大学）；Shandong University（山东大学）
University of Liverpool（利物浦大学）；Universidad Antonio de Nebrija（安东尼奥·德·内布里哈大学）；University of Electronic Science and Technology of China（电子科技大学）
This paper derives the optimal Bayesian processing of an out-of-sequence (OOS) set of measurements in continuous-time for multiple target tracking. We consider a multi-target system modelled in continuous time that is discretised at the time steps when we receive the measurements, which are distributed according to the standard point target model. All information about this system at the sampled time steps is provided by the posterior density on the set of all trajectories. This density can be computed via the continuous-discrete trajectory Poisson multi-Bernoulli mixture (TPMBM) filter. When …
GoalNet: Goal Areas Oriented Pedestrian Trajectory Prediction
Department of Electronic Engineering, National Taipei University of Technology（电子工程系，国立台北科技大学）；Department of Artificial Intelligence Technology, National Taipei University of Technology（人工智能技术系，国立台北科技大学）
