Robots that share an environment with humans may communicate their intent using a variety of different channels. Movement is one of these channels and, particularly in manipulation tasks, intent communication via movement is called legibility. It alters a robot's trajectory to make it intent expressive. Here we propose a novel evaluation method that improves the data efficiency of collected experimental data when benchmarking approaches generating such legible behavior. The primary novelty of the proposed method is that it uses trajectories that were generated independently of the framework being tested. This makes evaluation easier, enables N-way comparisons between approaches, and allows easier comparison across papers. We demonstrate the efficiency of the new evaluation method by comparing 10 legibility frameworks in 2 scenarios. The paper, thus, provides readers with (1) a novel approach to investigate and/or benchmark legibility, (2) an overview of existing frameworks, (3) an evaluation of 10 legibility frameworks (from 6 papers), and (4) evidence that viewing angle and trajectory progression matter when users evaluate the legibility of a motion.
Examining the legibility of humanoid robot arm movements in a pointing task
Faculty of Mathematics, Physics;Informatics, Comenius University Bratislava, Slovakia DIBRIS, University of Genoa, Genoa, Italy CONTACT, Italian Institute of Technology, Genoa, Italy Digital Health Dept., Nvision Systems \& Technologies, S.L., Barcelona, Spain Faculty of Education, University of Ljubljana, Slovenia
FLUID: A Fine-Grained Lightweight Urban Signalized-Intersection Dataset of Dense Conflict Trajectories
School of Intelligent Systems Engineering, Sun Yat-sen University(智能系统工程学院,中山大学);Guangdong Provincial Key Laboratory of Intelligent Transportation Systems(广东省智能交通系统重点实验室);Pengcheng Laboratory(鹏城实验室)
Evaluating Pedestrian Trajectory Prediction Methods with Respect to Autonomous Driving
Technical University of Munich(慕尼黑工业大学);School of Engineering & Design(工程与设计学院);Munich Institute of Robotics and Machine Intelligence (MIRMI)(慕尼黑机器人与机器智能研究所)
From Marginal to Joint Predictions: Evaluating Scene-Consistent Trajectory Prediction Approaches for Automated Driving
Pre-Development of Automated Driving, CARIAD SE(自动化驾驶前期开发,CARIAD SE);Technical University of Munich (TUM)(慕尼黑技术大学);Karlsruhe Institute of Technology (KIT)(卡尔斯鲁厄理工学院)