Ellipse Loss for Scene-Compliant Motion Prediction
Uber Advanced Technologies Group (ATG)（优步先进技术集团）；University of Florida（佛罗里达大学）
Henggang Cui, Hoda Shajari, Sai Yalamanchi, Nemanja Djuric
备注：
Henggang Cui and Hoda Shajari contributed equally to this work. Accepted for publication at IEEE International Conference on Robotics and Automation (ICRA) 2021
Motion prediction is a critical part of self-driving technology, responsible for inferring future behavior of traffic actors in autonomous vehicle's surroundings. In order to ensure safe and efficient operations, prediction models need to output accurate trajectories that obey the map constraints. In this paper, we address this task and propose a novel ellipse loss that allows the models to better reason about scene compliance and predict more realistic trajectories. Ellipse loss penalizes off-road predictions directly in a supervised manner, by projecting the output trajectories into the top-down map frame using a differentiable trajectory rasterizer module. Moreover, it takes into account actor dimensions and orientation, providing more direct training signals to the model. We applied ellipse loss to a recently proposed state-of-the-art joint detection-prediction model to showcase its benefits. Evaluation on large-scale autonomous driving data strongly indicates that the method allows for more accurate and more realistic trajectory predictions.
Aristotle University of Thessaloniki（塞萨洛尼基亚里士多德大学）
Semantic scene segmentation plays a critical role in a wide range of robotics applications, e.g., autonomous navigation. These applications are accompanied by specific computational restrictions, e.g., operation on low-power GPUs, at sufficient speed, and also for high-resolution input. Existing state-of-the-art segmentation models provide evaluation results under different setups and mainly considering high-power GPUs. In this paper, we investigate the behavior of the most successful semantic scene segmentation models, in terms of deployment (inference) speed, under various setups (GPUs, inpu…
Mimicking human ability to forecast future positions or interpret complex interactions in urban scenarios, such as streets, shopping malls or squares, is essential to develop socially compliant robots or self-driving cars. Autonomous systems may gain advantage on anticipating human motion to avoid collisions or to naturally behave alongside people. To foresee plausible trajectories, we construct an LSTM (long short-term memory)-based model considering three fundamental factors: people interactions, past observations in terms of previously crossed areas and semantics of surrounding space. Our m…
The paper describes a novel real-time algorithm for finding 3D geometric primitives (cylinders, cones and spheres) from 3D range data. In its core, it performs a fast model fitting with a model update in constant time (O(1)) for each new data point added to the model. We use a three stage approach.The first step inspects 1.5D sub spaces, to find ellipses. The next stage uses these ellipses as input by examining their neighborhood structure to form sets of candidates for the 3D geometric primitives. Finally, candidate ellipses are fitted to the geometric primitives. The complexity for point pro…
Predicting the trajectories of surrounding agents is an essential ability for autonomous vehicles navigating through complex traffic scenes. The future trajectories of agents can be inferred using two important cues: the locations and past motion of agents, and the static scene structure. Due to the high variability in scene structure and agent configurations, prior work has employed the attention mechanism, applied separately to the scene and agent configuration to learn the most salient parts of both cues. However, the two cues are tightly linked. The agent configuration can inform what part…
University of California, San Diego（加州大学圣迭戈分校）；University of Sao Paulo（圣保罗大学）
Forecasting long-term human motion is a challenging task due to the non-linearity, multi-modality and inherent uncertainty in future trajectories. The underlying scene and past motion of agents can provide useful cues to predict their future motion. However, the heterogeneity of the two inputs poses a challenge for learning a joint representation of the scene and past trajectories. To address this challenge, we propose a model based on grid representations to forecast agent trajectories. We represent the past trajectories of agents using binary 2-D grids, and the underlying scene as a RGB bird…
Inria（法国国家信息与自动化研究所）；Université de Lorraine（洛林大学）；LORIA（洛林计算机科学及其应用实验室）；CNRS（法国国家科学研究中心）
In this paper, we propose a method for initial camera pose estimation from just a single image which is robust to viewing conditions and does not require a detailed model of the scene. This method meets the growing need of easy deployment of robotics or augmented reality applications in any environments, especially those for which no accurate 3D model nor huge amount of ground truth data are available. It exploits the ability of deep learning techniques to reliably detect objects regardless of viewing conditions. Previous works have also shown that abstracting the geometry of a scene of object…
Stanford University（斯坦福大学）；University of Florida（佛罗里达大学）；University of Adelaide（阿德莱德大学）
This paper addresses the problem of path prediction for multiple interacting agents in a scene, which is a crucial step for many autonomous platforms such as self-driving cars and social robots. We present \textit{SoPhie}; an interpretable framework based on Generative Adversarial Network (GAN), which leverages two sources of information, the path history of all the agents in a scene, and the scene context information, using images of the scene. To predict a future path for an agent, both physical and social information must be leveraged. Previous work has not been successful to jointly model …
Inria（法国国家信息与自动化研究所）；Université de Lorraine（洛林大学）；CNRS（法国国家科学研究中心）
In this paper, we propose a method for coarse camera pose computation which is robust to viewing conditions and does not require a detailed model of the scene. This method meets the growing need of easy deployment of robotics or augmented reality applications in any environments, especially those for which no accurate 3D model nor huge amount of ground truth data are available. It exploits the ability of deep learning techniques to reliably detect objects regardless of viewing conditions. Previous works have also shown that abstracting the geometry of a scene of objects by an ellipsoid cloud a…
Controllable Diverse Sampling for Diffusion Based Motion Behavior Forecasting
Leibniz University Hannover（汉诺威莱布尼茨大学）；Institute of Cartography and Geoinformatics（制图与地理信息研究所）；University of Twente（特温特大学）；Faculty of Geo-Information Science and Earth Observation（地理信息科学与地球观测学院）
Tsinghua University（清华大学）；OpenAI；Peking University（北京大学）；KAUST（阿卜杜拉国王科技大学）；Mila（米拉）；University of California, Merced（加州大学默塞德分校）；Google Research（谷歌研究院）
Generating images from graph-structured inputs, such as scene graphs, is uniquely challenging due to the difficulty of aligning nodes and connections in graphs with objects and their relations in images. Most existing methods address this challenge by using scene layouts, which are image-like representations of scene graphs designed to capture the coarse structures of scene images. Because scene layouts are manually crafted, the alignment with images may not be fully optimized, causing suboptimal compliance between the generated images and the original scene graphs. To tackle this issue, we pr…
Traj-LLM: A New Exploration for Empowering Trajectory Prediction with Pre-trained Large Language Models
Beihang University（北京航空航天大学）；Research Institute for Frontier Science, Beihang University（北京航空航天大学前沿科学研究院）；Beijing University of Technology（北京工业大学）；College of Metropolitan Transportation, Beijing University of Technology（北京工业大学城市交通学院）
TrafficSignBench: Rule-Centric Closed-Loop Evaluation of Traffic-Sign Compliance in Autonomous Driving
Lomonosov Moscow State University（罗蒙诺索夫莫斯科国立大学）；FusionBrain Lab（FusionBrain实验室）；Moscow Institute of Physics and Technology（莫斯科物理技术学院）；HSE University（高等经济大学）；Innopolis University（因诺波利斯大学）；NUST MISIS（国立研究型技术大学莫斯科国立钢铁合金学院）
Uber Advanced Technologies Group（优步先进技术集团）；University of Toronto（多伦多大学）
Roads have well defined geometries, topologies, and traffic rules. While this has been widely exploited in motion planning methods to produce maneuvers that obey the law, little work has been devoted to utilize these priors in perception and motion forecasting methods. In this paper we propose to incorporate these structured priors as a loss function. In contrast to imposing hard constraints, this approach allows the model to handle non-compliant maneuvers when those happen in the real world. Safe motion planning is the end goal, and thus a probabilistic characterization of the possible future…
The University of Hong Kong（香港大学）；Fuzhou University（福州大学）
In this paper, we aim to forecast a future trajectory distribution of a moving agent in the real world, given the social scene images and historical trajectories. Yet, it is a challenging task because the ground-truth distribution is unknown and unobservable, while only one of its samples can be applied for supervising model learning, which is prone to bias. Most recent works focus on predicting diverse trajectories in order to cover all modes of the real distribution, but they may despise the precision and thus give too much credit to unrealistic predictions. To address the issue, we learn th…
Tsinghua University（清华大学）；Massachusetts Institute of Technology（麻省理工学院）
Predicting future motions of road participants is an important task for driving autonomously in urban scenes. Existing models excel at predicting marginal trajectories for single agents, yet it remains an open question to jointly predict scene compliant trajectories over multiple agents. The challenge is due to exponentially increasing prediction space as a function of the number of agents. In this work, we exploit the underlying relations between interacting agents and decouple the joint prediction problem into marginal prediction problems. Our proposed approach M2I first classifies interacti…
University of Padova（帕多瓦大学）；University of Modena and Reggio Emilia（摩德纳-雷焦艾米利亚大学）
Human trajectory forecasting is a key component of autonomous vehicles, social-aware robots and advanced video-surveillance applications. This challenging task typically requires knowledge about past motion, the environment and likely destination areas. In this context, multi-modality is a fundamental aspect and its effective modeling can be beneficial to any architecture. Inferring accurate trajectories is nevertheless challenging, due to the inherently uncertain nature of the future. To overcome these difficulties, recent models use different inputs and propose to model human intentions usin…
