Human safety is critical in applications involving close human-robot interactions (HRI) and is a key aspect of physical compatibility between humans and robots. While measures of human safety in HRI exist, these mainly target industrial settings involving robotic manipulators. Less attention has been paid to settings where mobile robots and humans share the space. This paper introduces a new robot-centered directional framework of human safety. It is particularly useful for evaluating mobile robots as they operate in environments populated by multiple humans. The framework integrates several key metrics, such as each human's relative distance, speed, and orientation. The core novelty lies in the framework's flexibility to accommodate different application requirements while allowing for both the robot-centered and external observer points of view. We instantiate the framework by using RGB-D based vision integrated with a deep learning-based human detection pipeline to yield a generalized safety index (GSI) that instantaneously assesses human safety. We evaluate GSI's capability of producing appropriate, robust, and fine-grained safety measures in real-world experimental scenarios and compare its performance with extant safety models.
Mobile service robots have become increasingly ubiquitous. However, these robots can pose potential accessibility issues and safety concerns to people with visual impairments (PVI). We sought to explore the challenges faced by PVI around mainstream mobile service robots and identify their needs. Seventeen PVI were interviewed about their experiences with three emerging robots: vacuum robots, delivery robots, and drones. We comprehensively investigated PVI's robot experiences by considering their different roles around robots -- direct users and bystanders. Our study highlighted participants' c…
Performing swift and agile maneuvers is essential for the safe operation of autonomous mobile robots. Moreover, the presence of time-delay restricts the response time of the system and hinders the safety performance. Thus, this paper proposes a modular and scalable safety-control design that utilizes the Smith predictor and barrier certificates to safely and consistently avoid obstacles with different footprints. The proposed solution includes a two-layer predictor to compensate for the time-delay in the servo-system and angle control loops. The proposed predictor configuration dramatically im…
Wayne State University（韦恩州立大学）；EPSI；Danieli Automation（达涅利自动化）
The design process and complexity of existing safety controls are heavily determined by the geometrical properties of the environment, which affects the proof of convergence, design scalability, performance robustness, and numerical efficiency of the control. Hence, this paper proposes a variable structure control to isolate the environment's geometrical complexity from the control structure. A super-twisting algorithm is used to achieve accurate trajectory tracking and robust safety control. The safety control is designed solely based on distance measurement. First, a nominal safety model for…
Mobile robots have gained increased importance within industrial tasks such as commissioning, delivery or operation in hazardous environments. The ability to autonomously navigate safely especially within dynamic environments, is paramount in industrial mobile robotics. Current navigation methods depend on preexisting static maps and are error-prone in dynamic environments. Furthermore, for safety reasons, they often rely on hand-crafted safety guidelines, which makes the system less flexible and slow. Visual based navigation and high level semantics bear the potential to enhance the safety of…
Purdue University（普渡大学）；Iowa State University（爱荷华州立大学）
The paper discusses how robots enable occupant-safe continuous protection for students when schools reopen. Conventionally, fixed air filters are not used as a key pandemic prevention method for public indoor spaces because they are unable to trap the airborne pathogens in time in the entire room. However, by combining the mobility of a robot with air filtration, the efficacy of cleaning up the air around multiple people is largely increased. A disinfection co-robot prototype is thus developed to provide continuous and occupant-friendly protection to people gathering indoors, specifically for …
Use-Inspired Mobile Robot to Improve Safety of Building Retrofit Workforce in Constrained Spaces
Institute for Experiential Robotics, Northeastern University（东北大学体验式机器人研究所）；Lincoln Laboratory, Massachusetts Institute of Technology（麻省理工学院林肯实验室）；Robotics Institute, Carnegie Mellon University（卡内基梅隆大学机器人研究所）；Amazon Scholar（亚马逊学者）；Northeastern University（东北大学）；Amazon.com, Inc.（亚马逊公司）
This review paper focuses on different aspects of perceived safety for a number of autonomous physical systems. This is a major aspect of robotics research, as more and more applications allow human and autonomous systems to share their space, with crucial implications both on safety and on its perception. The alternative terms used to express related concepts (e.g., psychological safety, trust, comfort, stress, fear, and anxiety) are listed and explained. Then, the available methods to assess perceived safety (i.e., questionnaires, physiological measurements, behavioral assessment, and direct…
Near-Field Perception for Safety Enhancement of Autonomous Mobile Robots in Manufacturing Environments
Department of Mechanical Engineering, University of Michigan, MI 48109, USA；Materials \& Manufacturing Systems Research, General Motors R \& D, Warren, MI 48092, USA
University Libre du Bruxelles（布鲁塞尔自由大学）；TU Delft（代尔夫特理工大学）
There is a growing demand for mobile robots to operate in more variable environments, where guaranteeing safe robot navigation is a priority, in addition to time performance. To achieve this, current solutions for local planning use a specific configuration tuned to the characteristics of the application environment. In this paper, we present an approach for developing quality models that can be used by a self-adaptation framework to adapt the local planner configuration at run-time based on the perceived environment. We contribute a definition of a safety model that predicts the safety of a n…
Reliability is a key factor for realizing safety guarantee of full autonomous robot systems. In this paper, we focus on reliability in mobile robot localization. Monte Carlo localization (MCL) is widely used for mobile robot localization. However, it is still difficult to guarantee its safety because there are no methods determining reliability for MCL estimate. This paper presents a novel localization framework that enables robust localization, reliability estimation, and quick re-localization, simultaneously. The presented method can be implemented using similar estimation manner to that of …
Autonomous warehouses with mobile, rack-carrying robots are starting to become commonplace, with systems such as Amazon's Kiva and Swisslog's CarryPick already implemented in functional warehouses. Such warehouses however still require human intervention for object picking and maintenance. In the European project SafeLog we are developing a safety-vest, used for safety-critical ranging and stopping of mobile robots, an improved planner that can handle large fleets of heterogeneous agents as well as an AR interaction system to navigate and support human workers in such automated environments. H…
We present an approach for visualizing mobile robots through an Augmented Reality headset when there is no line-of-sight visibility between the robot and the human. Three elements are visualized in Augmented Reality: 1) Robot's 3D model to indicate its position, 2) An arrow emanating from the robot to indicate its planned movement direction, and 3) A 2D grid to represent the ground plane. We conduct a user study with 18 participants, in which each participant are asked to retrieve objects, one at a time, from stations at the two sides of a T-junction at the end of a hallway where a mobile robo…
To operate with limited sensor horizons in unpredictable environments, autonomous robots use a receding-horizon strategy to plan trajectories, wherein they execute a short plan while creating the next plan. However, creating safe, dynamically-feasible trajectories in real time is challenging; and, planners must ensure persistent feasibility, meaning a new trajectory is always available before the previous one has finished executing. Existing approaches make a tradeoff between model complexity and planning speed, which can require sacrificing guarantees of safety and dynamic feasibility. This w…
School of Computer Science and Engineering（计算机科学与工程学院）
In a cyber-physical system such as an autonomous vehicle (AV), machine learning (ML) models can be used to navigate and identify objects that may interfere with the vehicle's operation. However, ML models are unlikely to make accurate decisions when presented with data outside their training distribution. Out-of-distribution (OOD) detection can act as a safety monitor for ML models by identifying such samples at run time. However, in safety critical systems like AVs, OOD detection needs to satisfy real-time constraints in addition to functional requirements. In this demonstration, we use a mob…
Compositional barrier functions are proposed in this paper to systematically compose multiple objectives for teams of mobile robots. The objectives are first encoded as barrier functions, and then composed using AND and OR logical operators. The advantage of this approach is that compositional barrier functions can provably guarantee the simultaneous satisfaction of all composed objectives. The compositional barrier functions are applied to the example of ensuring collision avoidance and static/dynamical graph connectivity of teams of mobile robots. The resulting composite safety and connectiv…
This paper extends the gap-based navigation technique in Potential Gap by guaranteeing safety for nonholonomic robots for all tiers of the local planner hierarchy, so called Safer Gap. The first tier generates a Bezier-based collision-free path through gaps. A subset of navigable free-space from the robot through a gap, called the keyhole, is defined to be the union of the largest collision-free disc centered on the robot and a trapezoidal region directed through the gap. It is encoded by a shallow neural network zeroing barrier function (ZBF). Nonlinear model predictive control (NMPC), with K…
University of Catania（卡塔尼亚大学）；STMicroelectronics（意法半导体）
Being able to estimate the traversability of the area surrounding a mobile robot is a fundamental task in the design of a navigation algorithm. However, the task is often complex, since it requires evaluating distances from obstacles, type and slope of terrain, and dealing with non-obvious discontinuities in detected distances due to perspective. In this paper, we present an approach based on deep learning to estimate and anticipate the traversing score of different routes in the field of view of an on-board RGB camera. The backbone of the proposed model is based on a state-of-the-art deep seg…
School of Electronics and Information Technology, Sun Yat-sen University（中山大学电子与信息工程学院）；School of Data and Computer Science, Sun Yat-sen University（中山大学数据与计算机学院）
Safety and tracking stability are crucial for safety-critical systems such as self-driving cars, autonomous mobile robots, industrial manipulators. To efficiently control safety-critical systems to ensure their safety and achieve tracking stability, accurate system dynamic models are usually required. However, accurate system models are not always available in practice. In this paper, a learning-based safety-stability-driven control (LBSC) algorithm is presented to guarantee the safety and tracking stability for nonlinear safety-critical systems subject to control input constraints under model…
Certain wheeled mobile robots e.g., electric wheelchairs, can operate through indirect joystick controls from users. Correct steering angle becomes essential when the user should determine the vehicle direction and velocity, in particular for differential wheeled vehicles since the vehicle velocity and direction are controlled with only two actuating wheels. This problem gets more challenging when complex curves should be realized by the user. A novel assistive controller with safety constraints is needed to address these problems. Also, the classic control methods mostly require the desired s…
Socially-aware Object Transportation by a Mobile Manipulator in Static Planar Environments with Obstacles
Computer Vision and Robotics Laboratory (VeRLab), Department of Computer Science, Universidade Federal de Minas Gerais（计算机视觉与机器人实验室（VeRLab）、计算机科学系、巴西矿务大学）
We address the problem of achieving persistent surveillance over an environment by using energy-constrained unmanned aerial vehicles (UAVs), which are supported by unmanned ground vehicles (UGVs) serving as mobile charging stations. Specifically, we plan the trajectories of all vehicles and the charging schedule of UAVs to minimize the long-term maximum age, where age is defined as the time between two consecutive visits to regions of interest in a partitioned environment. We introduce a scalable planning strategy based on 1) creating UAV- UGV teams, 2) decomposing the environment into optimal…
Koopman operator theory has served as the basis to extract dynamics for nonlinear system modeling and control across settings, including non-holonomic mobile robot control. There is a growing interest in research to derive robustness (and/or safety) guarantees for systems the dynamics of which are extracted via the Koopman operator. In this paper, we propose a way to quantify the prediction error because of noisy measurements when the Koopman operator is approximated via Extended Dynamic Mode Decomposition. We further develop an enhanced robot control strategy to endow robustness to a class of…
Tsinghua University（清华大学）；ETH Zurich（苏黎世联邦理工学院）；Harbin Institute of Technology（哈尔滨工业大学）
This paper presents an efficient and safe method to avoid static and dynamic obstacles based on LiDAR. First, point cloud is used to generate a real-time local grid map for obstacle detection. Then, obstacles are clustered by DBSCAN algorithm and enclosed with minimum bounding ellipses (MBEs). In addition, data association is conducted to match each MBE with the obstacle in the current frame. Considering MBE as an observation, Kalman filter (KF) is used to estimate and predict the motion state of the obstacle. In this way, the trajectory of each obstacle in the forward time domain can be param…
Imperial College London（伦敦帝国理工学院）；Indonesian Institute of Sciences - LIPI（印度尼西亚科学院）
In this work, we propose a novel mobile rescue robot equipped with an immersive stereoscopic teleperception and a teleoperation control. This robot is designed with the capability to perform safely a casualty-extraction procedure. We have built a proof-of-concept mobile rescue robot called ResQbot for the experimental platform. An approach called "loco-manipulation" is used to perform the casualty-extraction procedure using the platform. The performance of this robot is evaluated in terms of task accomplishment and safety by conducting a mock rescue experiment. We use a custom-made human-sized…
Point Cloud-Based Control Barrier Functions for Model Predictive Control in Safety-Critical Navigation of Autonomous Mobile Robots
School of Automation Science and Engineering, South China University of Technology（自动化科学与工程学院，华南理工大学）；Southern Marine Science and Engineering Guangdong Laboratory (Zhuhai)（南方海洋科学与工程 Guangdong 实验室（珠海））；Guangdong Academy of Safety Production and Emergency Management Science and Technology（广东安全生产与应急管理部门科学与技术研究院）
Assistance robots have gained widespread attention in various industries such as logistics and human assistance. The tasks of guiding or following a human in a crowded environment such as airports or train stations to carry weight or goods is still an open problem. In these use cases, the robot is not only required to intelligently interact with humans, but also to navigate safely among crowds. Thus, especially highly dynamic environments pose a grand challenge due to the volatile behavior patterns and unpredictable movements of humans. In this paper, we propose a Deep-Reinforcement-Learning-b…
