As social beings, much human behavior is predicated on social context - the ambient social state that includes cultural norms, social signals, individual preferences, etc. In this paper, we propose a socially-aware task and motion planning algorithm that considers social context to generate appropriate and effective plans in human social environments (HSEs). The key strength of our proposed approach is that it explicitly models how potential actions not only affect objective cost, but also transform the social context in which it plans and acts. We investigate strategies to limit the complexity of our algorithm, so that our planner will remain tractable for mobile platforms in complex HSEs like hospitals and factories. The planner will also consider the relative importance and urgency of its tasks, which it uses to determine when it is and is not appropriate to violate social expectations to achieve its objective. This social awareness will allow robots to understand a fundamental rule of society: just because something makes your job easier, does not make it the right thing to do!
To our knowledge, the proposed work is the first task and motion planning approach that supports socially intelligent robot policy for HSEs. Through this ongoing work, robots will be able to understand, respect, and leverage social context accomplish tasks both acceptably and effectively in HSEs.
University of Glasgow（格拉斯哥大学）；Heriot-Watt University（赫瑞-瓦特大学）；Idiap Research Institute（Idiap研究所）；LAAS-CNRS（法国国家科学研究中心分析与系统架构实验室）；SoftBank Robotics Europe（软银机器人欧洲公司）；VTT Technical Research Centre of Finland（芬兰技术研究中心）
In the EU-funded MuMMER project, we have developed a social robot designed to interact naturally and flexibly with users in public spaces such as a shopping mall. We present the latest version of the robot system developed during the project. This system encompasses audio-visual sensing, social signal processing, conversational interaction, perspective taking, geometric reasoning, and motion planning. It successfully combines all these components in an overarching framework using the Robot Operating System (ROS) and has been deployed to a shopping mall in Finland interacting with customers. In…
Heriot-Watt University（赫瑞-瓦特大学）；University of Glasgow（格拉斯哥大学）
We present an ongoing project that aims to develop a social robot to help children cope with painful and distressing medical procedures in a clinical setting. Our approach uses automated planning as a core component for action selection in order to generate plans that include physical, sensory, and social actions for the robot to use when interacting with humans. A key capability of our system is that the robot's behaviour adapts based on the affective state of the child patient. The robot must operate in a challenging physical and social environment where appropriate and safe interaction with…
In this work, we study how to build socially intelligent robots to assist people in their homes. In particular, we focus on assistance with online goal inference, where robots must simultaneously infer humans' goals and how to help them achieve those goals. Prior assistance methods either lack the adaptivity to adjust helping strategies (i.e., when and how to help) in response to uncertainty about goals or the scalability to conduct fast inference in a large goal space. Our NOPA (Neurally-guided Online Probabilistic Assistance) method addresses both of these challenges. NOPA consists of (1) an…
Mobile robots are increasingly populating homes, hospitals, shopping malls, factory floors, and other human environments. Human society has social norms that people mutually accept; obeying these norms is an essential signal that someone is participating socially with respect to the rest of the population. For robots to be socially compatible with humans, it is crucial for robots to obey these social norms. In prior work, we demonstrated a Socially-Aware Navigation (SAN) planner, based on Pareto Concavity Elimination Transformation (PaCcET), in a hallway scenario, optimizing two objectives so …
Social robots, also known as service or assistant robots, have been developed to improve the quality of human life in recent years. Socially assistive robots (SAR) are a special type of social robots that focus on providing support through social interaction. The design of socially capable and intelligent robots can vary, depending on the target user groups. In this work, I assess the effect of socially assistive robots' roles, functions, and communication approaches in the context of a social agent providing service or companionship to users with developmental disabilities. In this thesis, I …
ViLAM: Distilling Vision-Language Reasoning into Attention Maps for Social Robot Navigation
Dept. of Electrical and Computer Engineering, University of Maryland, College Park, MD, USA（电气与计算机工程系，马里兰大学，College Park, MD, USA）；Dept. of Computer Science, University of Maryland, College Park, MD, USA（计算机科学系，马里兰大学，College Park, MD, USA）；James Clark School of Engineering, University of Maryland, College Park, MD, USA（James Clark工程学院，马里兰大学，College Park, MD, USA）
This paper presents the framework for a novel Unified Socially-Aware Navigation (USAN) architecture and explains its need in Socially Assistive Robotics (SAR) applications. Our approach emphasizes interpersonal distance and how spatial communication can be used to build a unified planner for a human-robot collaborative environment. Socially-Aware Navigation (SAN) is vital for helping humans to feel comfortable and safe around robots; HRI studies have shown the importance of SAN transcends safety and comfort. SAN plays a crucial role in perceived intelligence, sociability and social capacity of…
Intelligent robots and machines are becoming pervasive in human populated environments. A desirable capability of these agents is to respond to goal-oriented commands by autonomously constructing task plans. However, such autonomy can add significant cognitive load and potentially introduce safety risks to humans when agents behave unexpectedly. Hence, for such agents to be helpful, one important requirement is for them to synthesize plans that can be easily understood by humans. While there exists previous work that studied socially acceptable robots that interact with humans in "natural ways…
Optimal-Horizon Social Robot Navigation in Heterogeneous Crowds
National Key Laboratory of Human-Machine Hybrid Augmented Intelligence（人机混合增强智能国家重点实验室）；Nation Engineering Research Center for Visual Information and Applications（视觉信息与应用国家工程研究中心）；Institute of Artificial Intelligence and Robotics（人工智能与机器人研究所）；Xi’an Jiaotong University（西安交通大学）
Human-robot collaborations have been recognized as an essential component for future factories. It remains challenging to properly design the behavior of those co-robots. Those robots operate in dynamic uncertain environment with limited computation capacity. The design objective is to maximize their task efficiency while guaranteeing safety. This paper discusses a set of design principles of a safe and efficient robot collaboration system (SERoCS) for the next generation co-robots, which consists of robust cognition algorithms for environment monitoring, efficient task planning algorithms for…
Massachusetts Institute of Technology（麻省理工学院）；IBM Thomas J. Watson Research Center（IBM托马斯·J·沃森研究中心）
For robotic vehicles to navigate safely and efficiently in pedestrian-rich environments, it is important to model subtle human behaviors and navigation rules (e.g., passing on the right). However, while instinctive to humans, socially compliant navigation is still difficult to quantify due to the stochasticity in people's behaviors. Existing works are mostly focused on using feature-matching techniques to describe and imitate human paths, but often do not generalize well since the feature values can vary from person to person, and even run to run. This work notes that while it is challenging t…
This paper develops a novel framework called Providers-Clients-Robots (PCR), applicable to socially assistive robots that support research on shared understanding in human-robot interactions. Providers, Clients, and Robots share an actionable and intuitive representation of the environment to create plans that best satisfy the combined needs of all parties. The plans are formed via interaction between the Client and the Robot based on a previously built multi-modal navigation graph. The explainable environmental representation in the form of a navigation graph is constructed collaboratively be…
Korea Advanced Institute of Science and Technology（韩国科学技术院）
In urban cities, with increasing acceptability of shared spaces used by pedestrians and personal mobility devices (PMDs), there is need for pragmatic socially ac-ceptable path planning and navigation management policies. Hence, we propose a socially acceptable global route planner and assess the legibility of the resulting global route. Our approach proposed for choosing global route avoids streets penetrating shared spaces and main routes with high probability of dense usage. The experimental study shows that socially acceptable routes can be effectively found with an average of 10 % incremen…
In this paper, we consider the problem of designing collision-free, dynamically feasible, and socially-aware trajectories for robots operating in environments populated by humans. We define trajectories to be social-aware if they do not interfere with humans in any way that causes discomfort. In this paper, discomfort is defined broadly and, depending on specific individuals, it can result from the robot being too close to a human or from interfering with human sight or tasks. Moreover, we assume that human feedback is a bandit feedback indicating a complaint or no complaint on the part of the…
University of California, Berkeley（加州大学伯克利分校）；Carnegie Mellon University（卡内基梅隆大学）
Planning under social interactions with other agents is an essential problem for autonomous driving. As the actions of the autonomous vehicle in the interactions affect and are also affected by other agents, autonomous vehicles need to efficiently infer the reaction of the other agents. Most existing approaches formulate the problem as a generalized Nash equilibrium problem solved by optimization-based methods. However, they demand too much computational resource and easily fall into the local minimum due to the non-convexity. Monte Carlo Tree Search (MCTS) successfully tackles such issues in …
National University of Defense Technology（国防科技大学）；National Innovation Institute of Defense Technology（国防科技创新研究院）
This study provides a systematic review of the recent advances in designing the intelligent tutoring robot (ITR), and summarises the status quo of applying artificial intelligence (AI) techniques. We first analyse the environment of the ITR and propose a relationship model for describing interactions of ITR with the students, the social milieu and the curriculum. Then, we transform the relationship model into the perception-planning-action model for exploring what AI techniques are suitable to be applied in the ITR. This article provides insights on promoting human-robot teaching-learning proc…
Socially Aware Motion Planning for Service Robots Using LiDAR and RGB-D Camera
Fulbright University Vietnam（富布赖特越南大学）；Vietnam National University Hanoi（越南国家大学河内分校）；University of Technology Sydney（悉尼科技大学）；Le Quy Don University（黎贵敦大学）
Follow-Bench: A Unified Motion Planning Benchmark for Socially-Aware Robot Person Following
Shenzhen Key Laboratory of Robotics and Computer Vision（深圳机器人与计算机视觉重点实验室）；Southern University of Science and Technology（南方科技大学）；Human-Robot Interfaces and Interaction Laboratory（人机交互实验室）；Istituto Italiano Di Tecnologia（意大利技术研究院）；Swiss Federal Technology Institute of Lausanne（洛桑联邦理工学院）
University of Eastern Finland（东芬兰大学）；Shenzhen Technology University（深圳技术大学）
We investigate and analyze principles of typical motion planning algorithms. These include traditional planning algorithms, supervised learning, optimal value reinforcement learning, policy gradient reinforcement learning. Traditional planning algorithms we investigated include graph search algorithms, sampling-based algorithms, and interpolating curve algorithms. Supervised learning algorithms include MSVM, LSTM, MCTS and CNN. Optimal value reinforcement learning algorithms include Q learning, DQN, double DQN, dueling DQN. Policy gradient algorithms include policy gradient method, actor-criti…
Department of Mechanical and Aerospace Engineering, Arizona State University（亚利桑那州立大学机械与航空航天工程系）；School of Computing, Informatics, and Decision Systems Engineering, Arizona State University（亚利桑那州立大学计算、信息与决策系统工程学院）；The Polytechnic School, Ira A. Fulton Schools of Engineering, Arizona State University（亚利桑那州立大学 Ira A. Fulton 工程学院理工学院）；Arizona State University（亚利桑那州立大学）
While intelligence of autonomous vehicles (AVs) has significantly advanced in recent years, accidents involving AVs suggest that these autonomous systems lack gracefulness in driving when interacting with human drivers. In the setting of a two-player game, we propose model predictive control based on social gracefulness, which is measured by the discrepancy between the actions taken by the AV and those that could have been taken in favor of the human driver. We define social awareness as the ability of an agent to infer such favorable actions based on knowledge about the other agent's intent, …
With the development of robotics and artificial intelligence field unceasingly thorough, path planning as an important field of robot calculation has been widespread concern. This paper analyzes the current development of robot and path planning algorithm and focuses on the advantages and disadvantages of the traditional intelligent path planning as well as the path planning. The problem of mobile robot path planning is studied by using ant colony algorithm, and it also provides some solving methods.
National University of Sciences And Technology (NUST)（国立科技大学）；Osaka University（大阪大学）
The sampling based motion planning algorithm known as Rapidly-exploring Random Trees (RRT) has gained the attention of many researchers due to their computational efficiency and effectiveness. Recently, a variant of RRT called RRT* has been proposed that ensures asymptotic optimality. Subsequently its bidirectional version has also been introduced in the literature known as Bidirectional-RRT* (B-RRT*). We introduce a new variant called Intelligent Bidirectional-RRT* (IB-RRT*) which is an improved variant of the optimal RRT* and bidirectional version of RRT* (B-RRT*) algorithms and is specially…
Washington University in St. Louis（圣路易斯华盛顿大学）；Duke University（杜克大学）；Stevens Institute of Technology（史蒂文斯理工学院）；University of Michigan（密歇根大学）；Northeastern University（东北大学）
Both pedestrian and robot comfort are of the highest priority whenever a robot is placed in an environment containing human beings. In the case of pedestrian-unaware mobile robots this desire for safety leads to the freezing robot problem, where a robot confronted with a large dynamic group of obstacles (such as a crowd of pedestrians) would determine all forward navigation unsafe causing the robot to stop in place. In order to navigate in a socially compliant manner while avoiding the freezing robot problem we are interested in understanding the flow of pedestrians in crowded scenarios. By tr…
University of Nevada, Las Vegas（内华达大学拉斯维加斯分校）；University of Nevada, Reno（内华达大学里诺分校）
We present a context classification pipeline to allow a robot to change its navigation strategy based on the observed social scenario. Socially-Aware Navigation considers social behavior in order to improve navigation around people. Most of the existing research uses different techniques to incorporate social norms into robot path planning for a single context. Methods that work for hallway behavior might not work for approaching people, and so on. We developed a high-level decision-making subsystem, a model-based context classifier, and a multi-objective optimization-based local planner to ac…
University of California at Berkeley（加州大学伯克利分校）；California PATH, University of California at Berkeley（加州大学伯克利分校加州PATH）
Autonomous cars have to navigate in dynamic environment which can be full of uncertainties. The uncertainties can come either from sensor limitations such as occlusions and limited sensor range, or from probabilistic prediction of other road participants, or from unknown social behavior in a new area. To safely and efficiently drive in the presence of these uncertainties, the decision-making and planning modules of autonomous cars should intelligently utilize all available information and appropriately tackle the uncertainties so that proper driving strategies can be generated. In this paper, …
Andhra University（安得拉大学）；National Institute of Science and Technology（国立科学技术学院）
Path planning of Robot is one of the challenging fields in the area of Robotics research. In this paper, we proposed a novel algorithm to find path between starting and ending position for an intelligent system. An intelligent system is considered to be a device/robot having an antenna connected with sensor-detector system. The proposed algorithm is based on Neural Network training concept. The considered neural network is Adapti ve to the knowledge bases. However, implementation of this algorithm is slightly expensive due to hardware it requires. From detailed analysis, it can be proved that …
Perspective-Shifted Neuro-Symbolic World Models: A Framework for Socially-Aware Robot Navigation
Institute for Systems and Robotics, Instituto Superior Técnico, Universidade de Lisboa（系统机器人研究所，理工学院，里斯本大学）；LAAS-CNRS, Artificial and Natural Intelligence Toulouse Institute (ANITI)（LAAS-CNRS，图卢兹人工智能研究所（ANITI））
Chalmers University of Technology（查尔姆斯理工大学）；Group Trucks Operation, Research & Technology Development (R&TD)（集团卡车运营研发部）
Systems based on the Robot Operating System (ROS) are easy to extend with new on-line algorithms and devices. However, there is relatively little support for coordinating a large number of heterogeneous sub-systems. In this paper we propose an architecture to model and control collaborative and intelligent automation systems in a hierarchical fashion.
SAGE: A Socially-Aware Generative Engine for Heterogeneous Multi-Agent Navigation
Guohao School, Tongji University（同济大学国豪书院）；Shanghai Research Institute for Intelligent Autonomous Systems（上海智能无人系统科学中心）；Tongji University（同济大学）；Western University（西安大略大学）；Aeromind Technology Co., Ltd.（深圳智元科技有限公司）；University at Buffalo-SUNY（纽约州立大学布法罗分校）
Safe and Socially Aware Multi-Robot Coordination in Multi-Human Social Care Settings
School of Computing and Communications, The Open University（奥本大学计算与通讯学院）；School of Electronics and Computer Science, University of Southampton（南安普顿大学电子与计算机科学学院）；School of Computer Science, University of Nottingham（诺丁汉大学计算机科学学院）
Milestones in Autonomous Driving and Intelligent Vehicles Part II: Perception and Planning
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；Waytous Ltd.（Waytous有限公司）；BNU-HKBU United International College（北京师范大学-香港浸会大学联合国际学院）；Hong Kong Baptist University（香港浸会大学）；Hunan University（湖南大学）；University of Cambridge（剑桥大学）；Xi’an Jiaotong University（西安交通大学）；Tsinghua University（清华大学）
Enhancing Social Robot Navigation with Integrated Motion Prediction and Trajectory Planning in Dynamic Human Environments
School of Information Science, Japan Advanced Institute of Science and Technology（日本先进科学技术学院信息科学学院）；University of Engineering and Technology, Vietnam National University Hanoi（越南国家大学河内工程技术大学）
Hainan University（海南大学）；University of Chinese Academy of Sciences（中国科学院大学）；Tianjin University（天津大学）
The agricultural irrigation system is closely related to agricultural production. There are some problems in nowadays agricultural irrigation system, such as poor mobility, imprecision and high price. To address these issues, an intelligent irrigation robot is designed and implemented in this work. The robot achieves precise irrigation by the irrigation path planning algorithm which is improved by Bayesian theory. In the proposed algorithm, we utilize as much information as possible to achieve full coverage irrigation in the complex agricultural environment. Besides, we propose the maximum ris…
Logic-Guided Socially-aware Robot Navigation World Model
School of Applied and Creative Computing, Purdue University（应用与创意计算学院，普渡大学）；Department of Computer Science, Purdue University（计算机科学系，普渡大学）；Department of Applied Artificial Intelligence, Sungkyunkwan University（应用人工智能系，成均馆大学）；Department of Computer Science and Department of Intelligent Systems Engineering, Indiana University Bloomington（计算机科学系和智能系统工程系，印第安纳大学布卢明顿分校）
Beihang University（北京航空航天大学）；University of California, Berkeley（加州大学伯克利分校）
As more and more autonomous vehicles (AVs) are being deployed on public roads, designing socially compatible behaviors for them is becoming increasingly important. In order to generate safe and efficient actions, AVs need to not only predict the future behaviors of other traffic participants, but also be aware of the uncertainties associated with such behavior prediction. In this paper, we propose an uncertain-aware integrated prediction and planning (UAPP) framework. It allows the AVs to infer the characteristics of other road users online and generate behaviors optimizing not only their own …
