A multi-modal framework to generate user intention distributions when operating a mobile vehicle is proposed in this work. The model learns from past observed trajectories and leverages traversability information derived from the visual surroundings to produce a set of future trajectories, suitable to be directly embedded into a perception-action shared control strategy on a mobile agent, or as a safety layer to supervise the prudent operation of the vehicle. We base our solution on a conditional Generative Adversarial Network with Long-Short Term Memory cells to capture trajectory distributions conditioned on past trajectories, further fused with traversability probabilities derived from visual segmentation with a Convolutional Neural Network. The proposed data-driven framework results in a significant reduction in error of the predicted trajectories (versus the ground truth) from comparable strategies in the literature (e.g. Social-GAN) that fail to account for information other than the agent's past history. Experiments were conducted on a dataset collected with a custom wheelchair model built onto the open-source urban driving simulator CARLA, proving also that the proposed framework can be used with a small, un-annotated dataset.
MUSON: A Reasoning-oriented Multimodal Dataset for Socially Compliant Navigation in Urban Environments
Graduate School of Information Science and Technology, Hokkaido University(北海道大学信息科学研究生院);Graduate School of Computer Science, George Mason University(乔治·马歇尔大学计算机科学研究生院)
ControlMTR: Control-Guided Motion Transformer with Scene-Compliant Intention Points for Feasible Motion Prediction
Department of Mechanical Engineering, National University of Singapore(新加坡国立大学机械工程系);College of Information Science and Engineering, Northeastern University(东北大学信息科学与工程学院);Moovita Pte Ltd(Moovita公司)
Aeolus Ocean -- A simulation environment for the autonomous COLREG-compliant navigation of Unmanned Surface Vehicles using Deep Reinforcement Learning and Maritime Object Detection
Institute of Informatics & Telecommunications, National Center for Scientific Research “Demokritos”(德谟克利特国家科学研究所信息与通信研究所)
EEG-Driven Intention Decoding: Offline Deep Learning Benchmarking on a Robotic Rover
Department of Computer Science, Imam Mohammad Ibn Saud Islamic University(计算机科学系,伊玛目穆罕默德·伊本·沙特伊斯兰大学);Department of Computer Science, King Saud University(计算机科学系,国王沙特大学)
Learning Robot Visual Navigation in Crowds via Intention-Aware Scene Representations
Shenzhen Key Laboratory of Robotics Perception and Intelligence, Department of Electronic and Electrical Engineering, SUSTech(南方科技大学电子与电气工程系深圳市机器人感知与智能重点实验室);Jiaxing Research Institute, SUSTech(南方科技大学嘉兴研究院);Research Institute of MA&EI, Peng Cheng Laboratory(鹏城实验室机器人与人工智能研究所)
MAction-SocialNav: Multi-Action Socially Compliant Navigation via Reasoning-enhanced Prompt Tuning
Graduate School of Information Science and Technology, Hokkaido University(信息科学与技术研究生院,北海道大学);Graduate School of Computer Science, George Mason University(计算机科学研究生院,乔治·梅森大学)
Interaction-Aware Model Predictive Decision-Making for Socially-Compliant Autonomous Driving in Mixed Urban Traffic Scenarios
Institute of Control Systems, Karlsruhe Institute for Technology(控制系统研究所,卡尔斯鲁厄技术学院);Wuerzburg Institute for Traffic Sciences GmbH(魏玛交通科学研究所有限公司);Faculty of Engineering - Mechanical Engineering Dept, University of Alberta(工程学院-机械工程系,阿尔伯塔大学)
MRReP: Mixed Reality-based Hand-drawn Reference Path Editing Interface for Mobile Robot Navigation
Graduate School of Information Science and Technology, The University of Osaka(大阪大学信息科学与技术研究生院);D3 Center, The University of Osaka(大阪大学D3中心);Graduate School of Maritime Sciences, Kobe University(神户大学海事科学研究生院)
WaterVideoQA: ASV-Centric Perception and Rule-Compliant Reasoning via Multi-Modal Agents
Thrust of Artificial Intelligence, The Hong Kong University of Science and Technology (Guangzhou)(香港理工大学(广州)人工智能研究所);Hubei Key Laboratory of Inland Shipping Technology (Wuhan University of Technology)(湖北内河航运技术重点实验室(武汉理工大学));School of Navigation, Wuhan University of Technology(武汉理工大学航海学院);School of Advanced Technology, Xi’an Jiaotong-Liverpool University(西安交通大学利物浦大学先进科技学院);School of Artificial Intelligence, Nanjing University(南京大学人工智能学院);School of Information Engineering, Yancheng Institute of Technology(盐城职业技术学院信息工程学院);School of Engineering, Stanford University(斯坦福大学工程学院);Centre for AI and Data Science Innovation and the School of Science and Engineering, James Cook University(詹姆斯库克大学人工智能与数据科学创新中心及科学与工程学院)
HCSG: Human-Centric Semantic-Geometric Reasoning for Vision-Language Navigation
The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州));Tsinghua University(清华大学);University of Science and Technology Beijing(北京科技大学);National University of Singapore(新加坡国立大学);Shanghai Jiao Tong University(上海交通大学)