When mobile robots maneuver near people, they run the risk of rudely blocking their paths; but not all people behave the same around robots. People that have not noticed the robot are the most difficult to predict. This paper investigates how mobile robots can generate acceptable paths in dynamic environments by predicting human behavior. Here, human behavior may include both physical and mental behavior, we focus on the latter. We introduce a simple safe interaction model: when a human seems unaware of the robot, it should avoid going too close. In this study, people around robots are detected and tracked using sensor fusion and filtering techniques. To handle uncertainties in the dynamic environment, a Partially-Observable Markov Decision Process Model (POMDP) is used to formulate a navigation planning problem in the shared environment. People's awareness of robots is inferred and included as a state and reward model in the POMDP. The proposed planner enables a robot to change its navigation plan based on its perception of each person's robot-awareness. As far as we can tell, this is a new capability. We conduct simulation and experiments using the Toyota Human Support Robot (HSR) to validate our approach. We demonstrate that the proposed framework is capable of running in real-time.
Noah’s Ark Lab, Huawei Technologies Canada, Ltd.（华为技术加拿大有限公司诺亚方舟实验室）；University of Alberta（阿尔伯塔大学）
We propose a method to tackle the problem of mapless collision-avoidance navigation where humans are present using 2D laser scans. Our proposed method uses ego-safety to measure collision from the robot's perspective while social-safety to measure the impact of our robot's actions on surrounding pedestrians. Specifically, the social-safety part predicts the intrusion impact of our robot's action into the interaction area with surrounding humans. We train the policy using reinforcement learning on a simple simulator and directly evaluate the learned policy in Gazebo and real robot tests. Experi…
University of Nevada, Las Vegas（内华达大学拉斯维加斯分校）；University of Nevada, Reno（内华达大学里诺分校）
We present a context classification pipeline to allow a robot to change its navigation strategy based on the observed social scenario. Socially-Aware Navigation considers social behavior in order to improve navigation around people. Most of the existing research uses different techniques to incorporate social norms into robot path planning for a single context. Methods that work for hallway behavior might not work for approaching people, and so on. We developed a high-level decision-making subsystem, a model-based context classifier, and a multi-objective optimization-based local planner to ac…
City University of Hong Kong（香港城市大学）；Baidu Research（百度研究院）；University of Maryland, College Park（马里兰大学帕克分校）
In this paper, we present a robotic navigation algorithm with natural language interfaces, which enables a robot to safely walk through a changing environment with moving persons by following human instructions such as "go to the restaurant and keep away from people". We first classify human instructions into three types: the goal, the constraints, and uninformative phrases. Next, we provide grounding for the extracted goal and constraint items in a dynamic manner along with the navigation process, to deal with the target objects that are too far away for sensor observation and the appearance …
WayFASTER: a Self-Supervised Traversability Prediction for Increased Navigation Awareness
University of Illinois at Urbana-Champaign (UIUC)（伊利诺伊大学厄巴纳-香槟分校）；Field Robotics Engineering and Science Hub (FRESH)（野外机器人工程与科学中心）；Illinois Autonomous Farm（伊利诺伊自主农场）
We present a micro aerial vehicle (MAV) system, built with inexpensive off-the-shelf hardware, for autonomously following trails in unstructured, outdoor environments such as forests. The system introduces a deep neural network (DNN) called TrailNet for estimating the view orientation and lateral offset of the MAV with respect to the trail center. The DNN-based controller achieves stable flight without oscillations by avoiding overconfident behavior through a loss function that includes both label smoothing and entropy reward. In addition to the TrailNet DNN, the system also utilizes vision mo…
Memory-Maze: Scenario Driven Visual Language Navigation Benchmark for Guiding Blind People
Waseda University（早稻田大学）；Miraikan - The National Museum of Emerging Science and Innovation（Mirai Kan - 国家新兴科学与创新博物馆）；Carnegie Mellon University（卡内基梅隆大学）
Learning from Demonstration with Failure Awareness for Safe Robot Navigation
Department of Mechanical Engineering, The Hong Kong Polytechnic University（香港理工大学机械工程系）；College of Information Science and Technology, Eastern Institute of Technology（东华理工大学信息科学学院）；School of Computer Science and Technology, University of Science and Technology of China（中国科学技术大学计算机科学与技术学院）；Department of Aeronautical and Aviation Engineering, The Hong Kong Polytechnic University（香港理工大学航空与航空工程系）
Following Is All You Need: Robot Crowd Navigation Using People As Planners
School of Electrical and Electronic Engineering, Nanyang Technological University（南洋理工大学电子与电气工程学院）；Robotics Institute, Carnegie Mellon University（卡内基梅隆大学机器人研究所）
University of North Carolina at Chapel Hill（北卡罗来纳大学教堂山分校）
We present a novel approach to automatically identify driver behaviors from vehicle trajectories and use them for safe navigation of autonomous vehicles. We propose a novel set of features that can be easily extracted from car trajectories. We derive a data-driven mapping between these features and six driver behaviors using an elaborate web-based user study. We also compute a summarized score indicating a level of awareness that is needed while driving next to other vehicles. We also incorporate our algorithm into a vehicle navigation simulation system and demonstrate its benefits in terms of…
Beyond Omakase: Designing Shared Control for Navigation Robots with Blind People
University of Maryland, College Park（马里兰大学学院公园分校）；IBM Research - Tokyo（IBM东京研究院）；Waseda University（早稻田大学）；Miraikan - The National Museum of Emerging Science and Innovation（未来馆（日本未来科学馆））
University of North Carolina at Chapel Hill（北卡罗来纳大学教堂山分校）；Tilburg University（蒂尔堡大学）；University of Maryland at College Park（马里兰大学帕克分校）
We present a Pedestrian Dominance Model (PDM) to identify the dominance characteristics of pedestrians for robot navigation. Through a perception study on a simulated dataset of pedestrians, PDM models the perceived dominance levels of pedestrians with varying motion behaviors corresponding to trajectory, speed, and personal space. At runtime, we use PDM to identify the dominance levels of pedestrians to facilitate socially-aware navigation for the robots. PDM can predict dominance levels from trajectories with ~85% accuracy. Prior studies in psychology literature indicate that when interactin…
People are proficient at communicating their intentions in order to avoid conflicts when navigating in narrow, crowded environments. In many situations mobile robots lack both the ability to interpret human intentions and the ability to clearly communicate their own intentions to people sharing their space. This work addresses the second of these points, leveraging insights about how people implicitly communicate with each other through observations of behaviors such as gaze to provide mobile robots with better social navigation skills. In a preliminary human study, the importance of gaze as a…
Brown University（布朗大学）；Rhode Island School of Design（罗德岛设计学院）
Our paper focuses on the research of the possibility for speech recognition intelligent agents to assist the elderly and disabled people's lives, to improve their life quality by utilizing cutting-edge technologies. After researching the attitude of elderly and disabled people toward the household agent, we propose a design framework: ESNI(Exploration, Segmentation, Navigation, Instruction) that apply to mobile agent, achieve some functionalities such as processing human commands, picking up a specified object, and moving an object to another location. The agent starts the exploration in an un…
We aim to understand how people assess human likeness in navigation produced by people and artificially intelligent (AI) agents in a video game. To this end, we propose a novel AI agent with the goal of generating more human-like behavior. We collect hundreds of crowd-sourced assessments comparing the human-likeness of navigation behavior generated by our agent and baseline AI agents with human-generated behavior. Our proposed agent passes a Turing Test, while the baseline agents do not. By passing a Turing Test, we mean that human judges could not quantitatively distinguish between videos of …
Johns Hopkins University（约翰斯·霍普金斯大学）；Johns Hopkins University Applied Physics Lab（约翰斯·霍普金斯大学应用物理实验室）；UC Berkeley（加州大学伯克利分校）
Human-aware robot navigation promises a range of applications in which mobile robots bring versatile assistance to people in common human environments. While prior research has mostly focused on modeling pedestrians as independent, intentional individuals, people move in groups; consequently, it is imperative for mobile robots to respect human groups when navigating around people. This paper explores learning group-aware navigation policies based on dynamic group formation using deep reinforcement learning. Through simulation experiments, we show that group-aware policies, compared to baseline…
Universidade de Tras-os-Montes e Alto Douro（特拉斯奥斯蒙特斯与阿尔托杜罗大学）；INESC TEC；Kathmandu University（加德满都大学）；Pokhara University（博卡拉大学）
There are many systems for indoor navigation specially built for visually impaired people but only some has good accuracy for navigation. While there are solutions like global navigation satellite systems for the localization outdoors, problems arise in urban scenarios and indoors due to insufficient or failed signal reception. To build a support system for navigation for visually impaired people, in this paper we present a comparison of indoor localization and navigation system, which performs continuous and real-time processing using commercially available systems (Beacons and Decawave) unde…
Exploring an unfamiliar indoor environment and avoiding obstacles is challenging for visually impaired people. Currently, several approaches achieve the avoidance of static obstacles based on the mapping of indoor scenes. To solve the issue of distinguishing dynamic obstacles, we propose an assistive system with an RGB-D sensor to detect dynamic information of a scene. Once the system captures an image, panoptic segmentation is performed to obtain the prior dynamic object information. With sparse feature points extracted from images and the depth information, poses of the user can be estimated…
Mobile robots are increasingly populating homes, hospitals, shopping malls, factory floors, and other human environments. Human society has social norms that people mutually accept; obeying these norms is an essential signal that someone is participating socially with respect to the rest of the population. For robots to be socially compatible with humans, it is crucial for robots to obey these social norms. In prior work, we demonstrated a Socially-Aware Navigation (SAN) planner, based on Pareto Concavity Elimination Transformation (PaCcET), in a hallway scenario, optimizing two objectives so …
Hong Kong University of Science and Technology（香港科技大学）；Carnegie Mellon University（卡内基梅隆大学）
In densely populated environments, socially compliant navigation is critical for autonomous robots as driving close to people is unavoidable. This manner of social navigation is challenging given the constraints of human comfort and social rules. Traditional methods based on hand-craft cost functions to achieve this task have difficulties to operate in the complex real world. Other learning-based approaches fail to address the naturalness aspect from the perspective of collective formation behaviors. We present an autonomous navigation system capable of operating in dense crowds and utilizing …
VLN-Cache: Enabling Token Caching for VLN Models with Visual/Semantic Dynamics Awareness
School of Computer Science, Peking University（北京大学计算机科学系）；School of Computer Science, China University of Geosciences (Wuhan)（中国地质大学（武汉）计算机科学系）；School of Artificial Intelligence and Automation, Huazhong University of Science and Technology（华中科技大学人工智能与自动化学院）；School of Electronics Engineering and Computer Science, Peking University（北京大学电子工程与计算机科学系）
Bi3: A Biplatform, Bicultural, Biperson Dataset for Social Robot Navigation
Department of Robotics, University of Michigan（机器人系，密歇根大学）；LAAS-CNRS, University of Toulouse（LAAS-CNRS，图卢兹大学）；INRIA, University of Lorraine（INRIA，洛林大学）
This paper studies how global dynamics and knowledge of high-level features can inform decision-making for robots in flow-like environments. Specifically, we investigate how coherent sets, an environmental feature found in these environments, inform robot awareness within these scenarios. The proposed approach is an online environmental feature generator which can be used for robot reasoning. We compute coherent sets online with techniques from machine learning and design frameworks for robot behavior that leverage coherent set features. We demonstrate the effectiveness of online methods over …
Risky and crowded environments (RCE) contain abstract sources of risk and uncertainty, which are perceived differently by humans, leading to a variety of behaviors. Thus, robots deployed in RCEs, need to exhibit diverse perception and planning capabilities in order to interpret other human agents' behavior and act accordingly in such environments. To understand this problem domain, we conducted a study to explore human path choices in RCEs, enabling better robotic navigational explainable AI (XAI) designs. We created a novel COVID-19 pandemic grocery shopping scenario which had time-risk trade…
Democritus University of Thrace（色雷斯德谟克利特大学）；Information Technologies Institute, The Centre for Research & Technology, Hellas（信息技术研究所，研究与技术中心，希腊）
This paper tackles the problem of positioning a swarm of UAVs inside a completely unknown terrain, having as objective to maximize the overall situational awareness. The situational awareness is expressed by the number and quality of unique objects of interest, inside the UAVs' fields of view. YOLOv3 and a system to identify duplicate objects of interest were employed to assign a single score to each UAVs' configuration. Then, a novel navigation algorithm, capable of optimizing the previously defined score, without taking into consideration the dynamics of either UAVs or environment, is propos…
University of Extremadura（埃斯特雷马杜拉大学）；Aston University（阿斯顿大学）；Pune Institute of Computer Technology（浦那计算机技术学院）
Autonomous navigation is a key skill for assistive and service robots. To be successful, robots have to minimise the disruption caused to humans while moving. This implies predicting how people will move and complying with social conventions. Avoiding disrupting personal spaces, people's paths and interactions are examples of these social conventions. This paper leverages Graph Neural Networks to model robot disruption considering the movement of the humans and the robot so that the model built can be used by path planning algorithms. Along with the model, this paper presents an evolution of t…
Navigation strategies that intentionally incorporate contact with humans (i.e. "contact-based" social navigation) in crowded environments are largely unexplored even though collision-free social navigation is a well studied problem. Traditional social navigation frameworks require the robot to stop suddenly or "freeze" whenever a collision is imminent. This paradigm poses two problems: 1) freezing while navigating a crowd may cause people to trip and fall over the robot, resulting in more harm than the collision itself, and 2) in very dense social environments where collisions are unavoidable,…
