We propose a Deep RObust Goal-Oriented trajectory prediction Network (DROGON) for accurate vehicle trajectory prediction by considering behavioral intentions of vehicles in traffic scenes. Our main insight is that the behavior (i.e., motion) of drivers can be reasoned from their high level possible goals (i.e., intention) on the road. To succeed in such behavior reasoning, we build a conditional prediction model to forecast goal-oriented trajectories with the following stages: (i) relational inference where we encode relational interactions of vehicles using the perceptual context; (ii) intention estimation to compute the probability distributions of intentional goals based on the inferred relations; and (iii) behavior reasoning where we reason about the behaviors of vehicles as trajectories conditioned on the intentions. To this end, we extend the proposed framework to the pedestrian trajectory prediction task, showing the potential applicability toward general trajectory prediction.
PlanTRansformer: Unified Prediction and Planning with Goal-conditioned Transformer
Department of Electrical Engineering and Information Technology, Intelligent Vehicles Lab (IVL), Munich University of Applied Science（电气工程与信息科技系，智能车辆实验室（IVL），慕尼黑应用科学大学）
Intention-Conditioned Long-Term Human Egocentric Action Forecasting
Technische Universität Wien (TU Wien)（维也纳技术大学）；Artificial Intelligence Graduate School (AIGS), Ulsan National Institute of Science and Technology (UNIST)（蔚山科学技术院人工智能研究生院）；Institute of Robotics and Mechatronics, German Aerospace Center（德国航空航天中心机器人与机电一体化研究所）
The University of Tokyo（东京大学）；National Institute of Informatics（国立信息学研究所）；Microsoft Research（微软研究院）
Image cropping has progressed tremendously under the data-driven paradigm. However, current approaches do not account for the intentions of the user, which is an issue especially when the composition of the input image is complex. Moreover, labeling of cropping data is costly and hence the amount of data is limited, leading to poor generalization performance of current algorithms in the wild. In this work, we take advantage of vision-language models as a foundation for creating robust and user-intentional cropping algorithms. By adapting a transformer decoder with a pre-trained CLIP-based dete…
University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）
The task of driver attention prediction has drawn considerable interest among researchers in robotics and the autonomous vehicle industry. Driver attention prediction can play an instrumental role in mitigating and preventing high-risk events, like collisions and casualties. However, existing driver attention prediction models neglect the distraction state and intention of the driver, which can significantly influence how they observe their surroundings. To address these issues, we present a new driver attention dataset, CoCAtt (Cognitive-Conditioned Attention). Unlike previous driver attentio…
University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）
The task of driver attention prediction has drawn considerable interest among researchers in robotics and the autonomous vehicle industry. Driver attention prediction can play an instrumental role in mitigating and preventing high-risk events, like collisions and casualties. However, existing driver attention prediction models neglect the distraction state and intention of the driver, which can significantly influence how they observe their surroundings. To address these issues, we present a new driver attention dataset, CoCAtt (Cognitive-Conditioned Attention). Unlike previous driver attentio…
Carnegie Mellon University（卡内基梅隆大学）；University of California, Berkeley（加利福尼亚大学伯克利分校）
For autonomous vehicles (AVs) to behave appropriately on roads populated by human-driven vehicles, they must be able to reason about the uncertain intentions and decisions of other drivers from rich perceptual information. Towards these capabilities, we present a probabilistic forecasting model of future interactions between a variable number of agents. We perform both standard forecasting and the novel task of conditional forecasting, which reasons about how all agents will likely respond to the goal of a controlled agent (here, the AV). We train models on real and simulated data to forecast …
Where Will They Go? Modelling Multimodal Pedestrian Manoeuvres from Ego-centric Videos
School of Computing Science, University of Glasgow（格拉斯哥大学计算机科学学院）；James Watt School of Engineering, University of Glasgow（格拉斯哥大学詹姆斯·瓦特工程学院）；Department of Computer Science, Durham University（杜伦大学计算机科学系）
Hong Kong University of Science and Technology（香港科技大学）；DJI Technology Company, Ltd.（大疆创新科技有限公司）
Motion prediction for traffic participants is essential for a safe and robust automated driving system, especially in cluttered urban environments. However, it is highly challenging due to the complex road topology as well as the uncertain intentions of the other agents. In this paper, we present a graph-based trajectory prediction network named the Dual Scale Predictor (DSP), which encodes both the static and dynamical driving context in a hierarchical manner. Different from methods based on a rasterized map or sparse lane graph, we consider the driving context as a graph with two layers, foc…
Applied AI Institute, Deakin University（迪肯大学应用人工智能研究所）
The prediction of humans' short-term trajectories has advanced significantly with the use of powerful sequential modeling and rich environment feature extraction. However, long-term prediction is still a major challenge for the current methods as the errors could accumulate along the way. Indeed, consistent and stable prediction far to the end of a trajectory inherently requires deeper analysis into the overall structure of that trajectory, which is related to the pedestrian's intention on the destination of the journey. In this work, we propose to model a hypothetical process that determines …
Data-driven methods open up unprecedented possibilities for maritime surveillance using Automatic Identification System (AIS) data. In this work, we explore deep learning strategies using historical AIS observations to address the problem of predicting future vessel trajectories with a prediction horizon of several hours. We propose novel sequence-to-sequence vessel trajectory prediction models based on encoder-decoder recurrent neural networks (RNNs) that are trained on historical trajectory data to predict future trajectory samples given previous observations. The proposed architecture combi…
Novel Diffusion Models for Multimodal 3D Hand Trajectory Prediction
IRMV Lab, the Department of Automation, Shanghai Jiao Tong University（IRMV实验室，自动化系，上海交通大学）；Meta Reality Labs（Meta现实实验室）；the Department of Electronic Engineering, Shanghai Jiao Tong University（电子工程系，上海交通大学）；the School of Information and Control Engineering, China University of Mining and Technology（信息与控制工程学院，中国矿业大学）；the College of Intelligence Science and Technology, National University of Defense Technology（智能科学与技术学院，国防科技大学）
