In this paper, we consider the problem of designing collision-free, dynamically feasible, and socially-aware trajectories for robots operating in environments populated by humans. We define trajectories to be social-aware if they do not interfere with humans in any way that causes discomfort. In this paper, discomfort is defined broadly and, depending on specific individuals, it can result from the robot being too close to a human or from interfering with human sight or tasks. Moreover, we assume that human feedback is a bandit feedback indicating a complaint or no complaint on the part of the robot trajectory that interferes with the humans, and it does not reveal any contextual information about the locations of the humans or the reason for a complaint. Finally, we assume that humans can move in the obstacle-free space and, as a result, human utility can change. We formulate this planning problem as an online optimization problem that minimizes the social value of the time-varying robot trajectory, defined by the total number of incurred human complaints. As the human utility is unknown, we employ zeroth order, or derivative-free, optimization methods to solve this problem, which we combine with off-the-shelf motion planners to satisfy the dynamic feasibility and collision-free specifications of the resulting trajectories. To the best of our knowledge, this is a new framework for socially-aware robot planning that is not restricted to avoiding collisions with humans but, instead, focuses on increasing the social value of the robot trajectories using only bandit human feedback.
Korea Advanced Institute of Science and Technology（韩国科学技术院）
In urban cities, with increasing acceptability of shared spaces used by pedestrians and personal mobility devices (PMDs), there is need for pragmatic socially ac-ceptable path planning and navigation management policies. Hence, we propose a socially acceptable global route planner and assess the legibility of the resulting global route. Our approach proposed for choosing global route avoids streets penetrating shared spaces and main routes with high probability of dense usage. The experimental study shows that socially acceptable routes can be effectively found with an average of 10 % incremen…
University of California, Berkeley（加州大学伯克利分校）；Carnegie Mellon University（卡内基梅隆大学）
Planning under social interactions with other agents is an essential problem for autonomous driving. As the actions of the autonomous vehicle in the interactions affect and are also affected by other agents, autonomous vehicles need to efficiently infer the reaction of the other agents. Most existing approaches formulate the problem as a generalized Nash equilibrium problem solved by optimization-based methods. However, they demand too much computational resource and easily fall into the local minimum due to the non-convexity. Monte Carlo Tree Search (MCTS) successfully tackles such issues in …
Socially Aware Motion Planning for Service Robots Using LiDAR and RGB-D Camera
Fulbright University Vietnam（富布赖特越南大学）；Vietnam National University Hanoi（越南国家大学河内分校）；University of Technology Sydney（悉尼科技大学）；Le Quy Don University（黎贵敦大学）
As social beings, much human behavior is predicated on social context - the ambient social state that includes cultural norms, social signals, individual preferences, etc. In this paper, we propose a socially-aware task and motion planning algorithm that considers social context to generate appropriate and effective plans in human social environments (HSEs). The key strength of our proposed approach is that it explicitly models how potential actions not only affect objective cost, but also transform the social context in which it plans and acts. We investigate strategies to limit the complexit…
Follow-Bench: A Unified Motion Planning Benchmark for Socially-Aware Robot Person Following
Shenzhen Key Laboratory of Robotics and Computer Vision（深圳机器人与计算机视觉重点实验室）；Southern University of Science and Technology（南方科技大学）；Human-Robot Interfaces and Interaction Laboratory（人机交互实验室）；Istituto Italiano Di Tecnologia（意大利技术研究院）；Swiss Federal Technology Institute of Lausanne（洛桑联邦理工学院）
In this paper, we consider a robot navigation problem in environments populated by humans. The goal is to determine collision-free and dynamically feasible trajectories that also maximize human satisfaction. This is because they may drive the robot close to humans that need help with their work or because they may keep the robot away from humans when it can interfere with human sight or work. In practice, human satisfaction is subjective and hard to describe mathematically. As a result, the planning problem we consider in this paper may lack important contextual information. To address this ch…
FZI Research Center for Information Technology（FZI信息技术研究中心）；Karlsruhe Institute of Technology（卡尔斯鲁厄理工学院）
Decision making under uncertainty can be framed as a partially observable Markov decision process (POMDP). Finding exact solutions of POMDPs is generally computationally intractable, but the solution can be approximated by sampling-based approaches. These sampling-based POMDP solvers rely on multi-armed bandit (MAB) heuristics, which assume the outcomes of different actions to be uncorrelated. In some applications, like motion planning in continuous spaces, similar actions yield similar outcomes. In this paper, we utilize variants of MAB heuristics that make Lipschitz continuity assumptions on…
Heriot-Watt University（赫瑞-瓦特大学）；University of Glasgow（格拉斯哥大学）
We present an ongoing project that aims to develop a social robot to help children cope with painful and distressing medical procedures in a clinical setting. Our approach uses automated planning as a core component for action selection in order to generate plans that include physical, sensory, and social actions for the robot to use when interacting with humans. A key capability of our system is that the robot's behaviour adapts based on the affective state of the child patient. The robot must operate in a challenging physical and social environment where appropriate and safe interaction with…
Massachusetts Institute of Technology（麻省理工学院）；IBM Thomas J. Watson Research Center（IBM托马斯·J·沃森研究中心）
For robotic vehicles to navigate safely and efficiently in pedestrian-rich environments, it is important to model subtle human behaviors and navigation rules (e.g., passing on the right). However, while instinctive to humans, socially compliant navigation is still difficult to quantify due to the stochasticity in people's behaviors. Existing works are mostly focused on using feature-matching techniques to describe and imitate human paths, but often do not generalize well since the feature values can vary from person to person, and even run to run. This work notes that while it is challenging t…
University of Nevada, Las Vegas（内华达大学拉斯维加斯分校）；University of Nevada, Reno（内华达大学里诺分校）
We present a context classification pipeline to allow a robot to change its navigation strategy based on the observed social scenario. Socially-Aware Navigation considers social behavior in order to improve navigation around people. Most of the existing research uses different techniques to incorporate social norms into robot path planning for a single context. Methods that work for hallway behavior might not work for approaching people, and so on. We developed a high-level decision-making subsystem, a model-based context classifier, and a multi-objective optimization-based local planner to ac…
Department of Mechanical and Aerospace Engineering, Arizona State University（亚利桑那州立大学机械与航空航天工程系）；School of Computing, Informatics, and Decision Systems Engineering, Arizona State University（亚利桑那州立大学计算、信息与决策系统工程学院）；The Polytechnic School, Ira A. Fulton Schools of Engineering, Arizona State University（亚利桑那州立大学 Ira A. Fulton 工程学院理工学院）；Arizona State University（亚利桑那州立大学）
While intelligence of autonomous vehicles (AVs) has significantly advanced in recent years, accidents involving AVs suggest that these autonomous systems lack gracefulness in driving when interacting with human drivers. In the setting of a two-player game, we propose model predictive control based on social gracefulness, which is measured by the discrepancy between the actions taken by the AV and those that could have been taken in favor of the human driver. We define social awareness as the ability of an agent to infer such favorable actions based on knowledge about the other agent's intent, …
Safe and Socially Aware Multi-Robot Coordination in Multi-Human Social Care Settings
School of Computing and Communications, The Open University（奥本大学计算与通讯学院）；School of Electronics and Computer Science, University of Southampton（南安普顿大学电子与计算机科学学院）；School of Computer Science, University of Nottingham（诺丁汉大学计算机科学学院）
We propose a novel approach for planning agents to compose abstract skills via observing and learning from historical interactions with the world. Our framework operates in a Markov state-space model via a set of actions under unknown pre-conditions. We formulate skills as high-level abstract policies that propose action plans based on the current state. Each policy learns new plans by observing the states' transitions while the agent interacts with the world. Such an approach automatically learns new plans to achieve specific intended effects, but the success of such plans is often dependent …
Enhancing Social Robot Navigation with Integrated Motion Prediction and Trajectory Planning in Dynamic Human Environments
School of Information Science, Japan Advanced Institute of Science and Technology（日本先进科学技术学院信息科学学院）；University of Engineering and Technology, Vietnam National University Hanoi（越南国家大学河内工程技术大学）
Mobile robots are increasingly populating homes, hospitals, shopping malls, factory floors, and other human environments. Human society has social norms that people mutually accept; obeying these norms is an essential signal that someone is participating socially with respect to the rest of the population. For robots to be socially compatible with humans, it is crucial for robots to obey these social norms. In prior work, we demonstrated a Socially-Aware Navigation (SAN) planner, based on Pareto Concavity Elimination Transformation (PaCcET), in a hallway scenario, optimizing two objectives so …
Logic-Guided Socially-aware Robot Navigation World Model
School of Applied and Creative Computing, Purdue University（应用与创意计算学院，普渡大学）；Department of Computer Science, Purdue University（计算机科学系，普渡大学）；Department of Applied Artificial Intelligence, Sungkyunkwan University（应用人工智能系，成均馆大学）；Department of Computer Science and Department of Intelligent Systems Engineering, Indiana University Bloomington（计算机科学系和智能系统工程系，印第安纳大学布卢明顿分校）
SAGE: A Socially-Aware Generative Engine for Heterogeneous Multi-Agent Navigation
Guohao School, Tongji University（同济大学国豪书院）；Shanghai Research Institute for Intelligent Autonomous Systems（上海智能无人系统科学中心）；Tongji University（同济大学）；Western University（西安大略大学）；Aeromind Technology Co., Ltd.（深圳智元科技有限公司）；University at Buffalo-SUNY（纽约州立大学布法罗分校）
Zhejiang Lab（之江实验室）；School of Computer Science and Engineering, Tianjin University of Technology（天津理工大学计算机科学与工程学院）
The guiding task of a mobile robot requires not only human-aware navigation, but also appropriate yet timely interaction for active instruction. State-of-the-art tour-guide models limit their socially-aware consideration to adapting to users' motion, ignoring the interactive behavior planning to fulfill the communicative demands. We propose a multi-behavior planning framework based on Monte Carlo Tree Search to better assist users to understand confusing scene contexts, select proper paths and timely arrive at the destination. To provide proactive guidance, we construct a sampling-based probab…
This paper presents the framework for a novel Unified Socially-Aware Navigation (USAN) architecture and explains its need in Socially Assistive Robotics (SAR) applications. Our approach emphasizes interpersonal distance and how spatial communication can be used to build a unified planner for a human-robot collaborative environment. Socially-Aware Navigation (SAN) is vital for helping humans to feel comfortable and safe around robots; HRI studies have shown the importance of SAN transcends safety and comfort. SAN plays a crucial role in perceived intelligence, sociability and social capacity of…
We introduce and show preliminary results of a fast randomized method that finds a set of K paths lying in distinct homotopy classes. We frame the path planning task as a graph search problem, where the navigation graph is based on a Voronoi diagram. The search is biased by a cost function derived from the social force model that is used to generate and select the paths. We compare our method to Yen's algorithm, and empirically show that our approach is faster to find a subset of homotopy classes. Furthermore our approach computes a set of more diverse paths with respect to the baseline while …
In unstructured urban canals, regulation-aware interactions with other vessels are essential for collision avoidance and social compliance. In this paper, we propose a regulations aware motion planning framework for Autonomous Surface Vessels (ASVs) that accounts for dynamic and static obstacles. Our method builds upon local model predictive contouring control (LMPCC) to generate motion plans satisfying kino-dynamic and collision constraints in real-time while including regulation awareness. To incorporate regulations in the planning stage, we propose a cost function encouraging compliance wit…
National University of Singapore（新加坡国立大学）；Yinson；MooVita
Accurately predicting interactive road agents' future trajectories and planning a socially compliant and human-like trajectory accordingly are important for autonomous vehicles. In this paper, we propose a planning-centric prediction neural network, which takes surrounding agents' historical states and map context information as input, and outputs the joint multi-modal prediction trajectories for surrounding agents, as well as a sequence of control commands for the ego vehicle by imitation learning. An agent-agent interaction module along the time axis is proposed in our network architecture t…
Both pedestrian and robot comfort are of the highest priority whenever a robot is placed in an environment containing human beings. In the case of pedestrian-unaware mobile robots this desire for safety leads to the freezing robot problem, where a robot confronted with a large dynamic group of obstacles (such as a crowd of pedestrians) would determine all forward navigation unsafe causing the robot to stop in place. In order to navigate in a socially compliant manner while avoiding the freezing robot problem we are interested in understanding the flow of pedestrians in crowded scenarios. By tr…
University of Glasgow（格拉斯哥大学）；Heriot-Watt University（赫瑞-瓦特大学）；Idiap Research Institute（Idiap研究所）；LAAS-CNRS（法国国家科学研究中心分析与系统架构实验室）；SoftBank Robotics Europe（软银机器人欧洲公司）；VTT Technical Research Centre of Finland（芬兰技术研究中心）
In the EU-funded MuMMER project, we have developed a social robot designed to interact naturally and flexibly with users in public spaces such as a shopping mall. We present the latest version of the robot system developed during the project. This system encompasses audio-visual sensing, social signal processing, conversational interaction, perspective taking, geometric reasoning, and motion planning. It successfully combines all these components in an overarching framework using the Robot Operating System (ROS) and has been deployed to a shopping mall in Finland interacting with customers. In…
Vision-Language Models for Deployable Social Robot Navigation: Bridging Semantic Reasoning and Low-Level Control
Graduate School of Information Science and Technology, Hokkaido University（弘前大学信息科学与技术研究生院）；Graduate School of Information Science and Technology, The University of Tokyo（东京大学信息科学与技术研究生院）
Perspective-Shifted Neuro-Symbolic World Models: A Framework for Socially-Aware Robot Navigation
Institute for Systems and Robotics, Instituto Superior Técnico, Universidade de Lisboa（系统机器人研究所，理工学院，里斯本大学）；LAAS-CNRS, Artificial and Natural Intelligence Toulouse Institute (ANITI)（LAAS-CNRS，图卢兹人工智能研究所（ANITI））
This paper develops a novel framework called Providers-Clients-Robots (PCR), applicable to socially assistive robots that support research on shared understanding in human-robot interactions. Providers, Clients, and Robots share an actionable and intuitive representation of the environment to create plans that best satisfy the combined needs of all parties. The plans are formed via interaction between the Client and the Robot based on a previously built multi-modal navigation graph. The explainable environmental representation in the form of a navigation graph is constructed collaboratively be…
Beihang University（北京航空航天大学）；University of California, Berkeley（加州大学伯克利分校）
As more and more autonomous vehicles (AVs) are being deployed on public roads, designing socially compatible behaviors for them is becoming increasingly important. In order to generate safe and efficient actions, AVs need to not only predict the future behaviors of other traffic participants, but also be aware of the uncertainties associated with such behavior prediction. In this paper, we propose an uncertain-aware integrated prediction and planning (UAPP) framework. It allows the AVs to infer the characteristics of other road users online and generate behaviors optimizing not only their own …
SAP-CoPE: Social-Aware Planning using Cooperative Pose Estimation with Infrastructure Sensor Nodes
Mechanical and Mechatronics Eng. Department, University of Waterloo（机械与机电工程系，滑铁库大学）；Department of Data and Systems Engineering, University of Hong Kong（数据与系统工程系，香港大学）；Mechanical Engineering Department, University of Alberta（机械工程系，阿尔伯塔大学）
ViLAM: Distilling Vision-Language Reasoning into Attention Maps for Social Robot Navigation
Dept. of Electrical and Computer Engineering, University of Maryland, College Park, MD, USA（电气与计算机工程系，马里兰大学，College Park, MD, USA）；Dept. of Computer Science, University of Maryland, College Park, MD, USA（计算机科学系，马里兰大学，College Park, MD, USA）；James Clark School of Engineering, University of Maryland, College Park, MD, USA（James Clark工程学院，马里兰大学，College Park, MD, USA）
Nanyang Technological University（南洋理工大学）；Institute for Infocomm Research, Agency for Science, Technology and Research（新加坡科技研究局资讯通信研究院）
Tactile sensors have been introduced to a wide range of robotic tasks such as robot manipulation to mimic the sense of human touch. However, there has only been a few works that integrate tactile sensing into robot navigation. This paper describes a navigation system which allows robots to operate in crowded human-dense environments and behave with socially acceptable reactions by utilizing semantic and force information collected by embedded tactile sensors, RGB-D camera and LiDAR. Compliance control is implemented based on artificial potential fields considering not only laser scan but also …
University of British Columbia（不列颠哥伦比亚大学）；Monash University（莫纳什大学）
Mobile robots have recently been deployed in public spaces such as shopping malls, airports, and urban sidewalks. Most of these robots are designed with human-aware motion planning capabilities but are not designed to communicate with pedestrians. Pedestrians that encounter these robots without prior understanding of the robots' behaviour can experience discomfort, confusion, and delayed social acceptance. In this work we designed and evaluated nonverbal robot motion legibility cues, which communicate a mobile robot's motion intention to pedestrians. We compared a motion legibility cue using P…
Learning Social Heuristics for Human-Aware Path Planning
Department of Electronics and Telecommunications (DET) Politecnico di Torino（电子工程与电信学院（DET）理工学院）；Department of Informatics King’s College London（信息学院伦敦国王学院）
This paper explores the problem of path planning under uncertainty. Specifically, we consider online receding horizon based planners that need to operate in a latent environment where the latent information can be modeled via Gaussian Processes. Online path planning in latent environments is challenging since the robot needs to explore the environment to get a more accurate model of latent information for better planning later and also achieves the task as quick as possible. We propose UCB style algorithms that are popular in the bandit settings and show how those analyses can be adapted to th…
Intelligent robots and machines are becoming pervasive in human populated environments. A desirable capability of these agents is to respond to goal-oriented commands by autonomously constructing task plans. However, such autonomy can add significant cognitive load and potentially introduce safety risks to humans when agents behave unexpectedly. Hence, for such agents to be helpful, one important requirement is for them to synthesize plans that can be easily understood by humans. While there exists previous work that studied socially acceptable robots that interact with humans in "natural ways…
