In this paper we present a framework for combining deep learning-based road detection, particle filters, and Model Predictive Control (MPC) to drive aggressively using only a monocular camera, IMU, and wheel speed sensors. This framework uses deep convolutional neural networks combined with LSTMs to learn a local cost map representation of the track in front of the vehicle. A particle filter uses this dynamic observation model to localize in a schematic map, and MPC is used to drive aggressively using this particle filter based state estimate. We show extensive real world testing results, and demonstrate reliable operation of the vehicle at the friction limits on a complex dirt track. We reach speeds above 27 mph (12 m/s) on a dirt track with a 105 foot (32m) long straight using our 1:5 scale test vehicle. A video of these results can be found at https://www.youtube.com/watch?v=5ALIK-z-vUg
Learning Interpretable End-to-End Vision-Based Motion Planning for Autonomous Driving with Optical Flow Distillation
The Hong Kong University of Science and Technology(香港科技大学);The Hong Kong Polytechnic University(香港理工大学);Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences(中国科学院深圳先进技术研究院)
A Kalman Filter-Based Disturbance Observer for Steer-by-Wire Systems
Professorship of Autonomous Vehicle Systems, Technical University of Munich(自动驾驶车辆系统教授职位,慕尼黑技术大学);Munich Institute of Robotics and Machine Intelligence (MIRMI)(慕尼黑机器人与机器智能研究所(MIRMI));Chair of Automotive Engineering, Institute of Land and Sea Transport, Technical University of Berlin(汽车工程教授职位,陆海运输研究所,柏林技术大学)
PESAO: Psychophysical Experimental Setup for Active Observers
York University(约克大学);Department of Electrical Engineering and Computer Science(电气工程与计算机科学系)
大多数过去和现在的计算机视觉研究都涉及被动观察的数据。然而,人类是实验室之外的积极观察者;他们探索,搜索,选择看什么以及如何看。尽管如此,如何准确地在人类中进行主动观察,以便它可以为主动计算机视觉系统的设计提供信息,这是一个悬而未决的问题。PESAO旨在研究3D世界中的主动视觉观察。我们的目标是建立一个实验装置,用于各种主动感知任务,其中考虑到人类受试者(主动观察者)能够跟踪头部和凝视。虽然许多研究探索人类的表现,但通常,他们使用2D描绘的线条画,没有主动观察者参与。PESAO使我们能够将许多研究带到三维世界,甚至涉及主动观察者。在我们的实例中,它跨越400 cm x 300 cm的区域,可以以120 Hz的频率跟踪活跃的观察者。此外,PESAO提供跟踪和记录6D头部运动,凝视,眼睛运动类型,第一人称视频,头戴式IMU传感器,鸟瞰视频和实验者笔记。所有这些都以微秒级的分辨率同步。
LightEMMA: A Longitudinal Evaluation of Vision-Language Models for Autonomous Driving
Department of Civil and Environmental Engineering, University of Michigan(土木与环境工程系,密歇根大学);University of Michigan Transportation Research Institute(密歇根大学交通研究所)
Vision-based DRL Autonomous Driving Agent with Sim2Real Transfer
Chair of Econometrics and Statistics, esp. in the Transport Sector, Technische Universität Dresden(德累斯顿工业大学交通领域计量经济学与统计学讲席);Center for Scalable Data Analytics and Artificial Intelligence (ScaDS.AI) Dresden/Leipzig(德累斯顿/莱ipzig可扩展数据分析与人工智能中心(ScaDS.AI))
AppleVLM: End-to-end Autonomous Driving with Advanced Perception and Planning-Enhanced Vision-Language Models
Shenzhen Key Lab for Advanced Motion Control and Modern Automation Equipments(深圳先进运动控制系统与现代自动化装备重点实验室);Guangdong Provincial Key Laboratory of Intelligent Morphing Mechanisms and Adaptive Robotics(广东省智能变形机制与适应机器人省重点实验室);School of Intelligence Science and Engineering(智能科学与工程学院);Harbin Institute of Technology(哈尔滨工业大学);National Key Laboratory of Smart Farm Technologies and Systems(国家智能农业技术与系统重点实验室);Autonomous Driving Center(自动驾驶中心);Shanghai Utopilot Technology Co.Ltd.(上海驭目科技有限公司)
Reasoning-VLA: A Fast and General Vision-Language-Action Reasoning Model for Autonomous Driving
Lanzhou University(兰州大学);National University of Singapore(新加坡国立大学);University of Science and Technology of China(中国科学技术大学);Tsinghua University(清华大学);University of New South Wales(新南威尔士大学)
GRI: General Reinforced Imitation and its Application to Vision-Based Autonomous Driving
Mines Paris - PSL University(巴黎矿业学院-PSL大学);Valeo DAR(法雷奥DAR)
深度强化学习(DRL)已被证明对自动驾驶和机器人等几个复杂的决策应用是有效的。然而,DRL因其高样品复杂性和缺乏稳定性而受到限制。先前的知识,例如专家演示,通常是可用的,但很难利用来缓解这些问题。在本文中,我们提出了通用强化模仿(GRI),这是一种新的方法,它结合了探索和专家数据的好处,并且可以直接在任何非策略RL算法上实现。我们做一个简单的假设:专家的论证可以被看作是完美的数据,其基础政策得到了持续的高回报。基于这一假设,GRI引入了离线演示代理的概念。该代理发送专家数据,这些数据与来自在线RL探索代理的经验同时且不可分割地处理。我们表明,我们的方法能够在城市环境中实现基于视觉的自动驾驶的重大改进。我们进一步验证了GRI方法的Mujoco连续控制任务与不同的政策外RL算法。我们的方法在CARLA排行榜上排名第一,比之前最先进的World on Rails高出17%。
Application of Vision-Language Model to Pedestrians Behavior and Scene Understanding in Autonomous Driving
ECE Department(电子工程系);Carnegie Mellon University(卡内基梅隆大学);Computer Science Department(计算机科学系);Columbia University(哥伦比亚大学);Rotman School of Management(罗特曼管理学院);University of Toronto(多伦多大学);Department of Statistics(统计学系);George Washington University(乔治华盛顿大学);Department of Computer Science(计算机科学系);San Francisco State University(旧金山州立大学)
DRIVE: Dependable Robust Interpretable Visionary Ensemble Framework in Autonomous Driving
Tongji University(同济大学);King Abdullah University of Science and Technology(阿卜杜拉国王科技大学);The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州));University of Macau(澳门大学)
VECTOR-Drive: Tightly Coupled Vision-Language and Trajectory Expert Routing for End-to-End Autonomous Driving
College of Automotive Engineering, Jilin University(吉林大学汽车工程学院);The National Key Laboratory of Automotive Chassis Integration and Bionics, Jilin University(吉林大学汽车底盘集成与生物力学国家级重点实验室);ReeFocus AI Technology(ReeFocus人工智能技术)
DINO Pre-training for Vision-based End-to-end Autonomous Driving
Vilnius University Institute of Data Science and Digital Technologies(维尔纽斯大学数据科学与数字技术研究所);Research Institute of Natural and Technological Sciences, Vytautas Magnus University(维陶塔斯马格努斯大学自然与技术科学研究所)