传统的机器人任务规划AI规划方法需要经过符号编码的领域描述。尽管这类方法在定义明确的场景中功能强大，并且对人类可解释，但建立这种描述需要大量工作。与此不同，人类解决大多数日常规划任务时依靠直觉，使用对不同规划步骤的心理意象。本文提出，在只要求有限执行精度的情况下，同样的方法也可用于机器人。在本研究中，我们提出一种新颖的亚符号方法，称为用于规划的模拟心理意象（Simulated Mental Imagery for Planning，SiMIP），它由在“想象”图像上执行的感知、模拟动作、成功检查和重新规划组成。我们表明，通过结合常规卷积神经网络和生成对抗网络，可以以算法上可靠的方式实现基于心理意象的规划。借助该方法，机器人获得了利用初始现有场景生成动作规划的能力，而无需符号化的领域描述；同时，与作为另一种亚符号方法的深度强化学习不同，所生成的规划仍保持人类可解释性。我们针对一个必须将不同物体正确放入不同目标槽位的装箱问题，利用真实场景创建了一个数据集，从而能够量化该算法的效率和成功率。
英文摘要：
Traditional AI-planning methods for task planning in robotics require a symbolically encoded domain description. While powerful in well-defined scenarios, as well as human-interpretable, setting this up requires substantial effort. Different from this, most everyday planning tasks are solved by humans intuitively, using mental imagery of the different planning steps. Here we suggest that the same approach can be used for robots, too, in cases which require only limited execution accuracy. In the current study, we propose a novel sub-symbolic method called Simulated Mental Imagery for Planning (SiMIP), which consists of perception, simulated action, success-checking and re-planning performed on 'imagined' images. We show that it is possible to implement mental imagery-based planning in an algorithmically sound way by combining regular convolutional neural networks and generative adversarial networks. With this method, the robot acquires the capability to use the initially existing scene to generate action plans without symbolic domain descriptions, while at the same time plans remain human-interpretable, different from deep reinforcement learning, which is an alternative sub-symbolic approach. We create a dataset from real scenes for a packing problem of having to correctly place different objects into different target slots. This way efficiency and success rate of this algorithm could be quantified.
Institute of Data Science and Digital Technologies, Vilnius University（维尔纽斯大学数据科学与数字技术研究所）
Purpose: This paper presents a new dataset of Aerial Imagery from Robotics simulator (abbr. AIR). AIR dataset aims to provide a starting point for localization system development and to become a typical benchmark for accuracy comparison of map-based localization algorithms, visual odometry, and SLAM for high altitude flights. Design/methodology/approach: The presented dataset contains over 100 thousand aerial images captured from Gazebo robotics simulator using orthophoto maps as a ground plane. Flights with 3 different trajectories are performed on maps from urban and forest environment at di…
Humans form mental images of 3D scenes to support counterfactual imagination, planning, and motor control. Our abilities to predict the appearance and affordance of the scene from previously unobserved viewpoints aid us in performing manipulation tasks (e.g., 6-DoF kitting) with a level of ease that is currently out of reach for existing robot learning frameworks. In this work, we aim to build artificial systems that can analogously plan actions on top of imagined images. To this end, we introduce Mental Imagery for Robotic Affordances (MIRA), an action reasoning framework that optimizes actio…
ShotcreteDepth: A Bi-modal Dataset for Robust Robotic Depth Perception in Shotcrete Construction Environments
Technical University of Denmark, Department of Electrical and Photonics Engineering（丹麦技术大学电气与光子工程系）；Pioneer Centre for AI（人工智能先锋中心）；Christiansen & Essenbæk A/S（Christiansen & Essenbæk 公司）
This chapter discusses the existing and future use of robotics and intelligent sensing technology in mental health care. While the use of this technology is nascent in mental health care, it represents a potentially useful tool in the practitioner's toolbox. The goal of this chapter is to provide a brief overview of the field, discuss the recent use of robotics technology in mental health care practice, explore some of the design issues and ethical issues of using robots in this space, and finally to explore the potential of emerging technology.
Robotic Grasping and Placement Controlled by EEG-Based Hybrid Visual and Motor Imagery
Institute of Science and Technology for Brain-inspired Intelligence (ISTBI), Fudan University（复旦大学类脑智能科学与技术研究院）；School of Data Science, Fudan University（复旦大学大数据学院）；Institute of Trustworthy Embodied AI, Fudan University（复旦大学可信具身智能研究院）；Shanghai Key Laboratory of Multimodal Embodied AI（上海市多模态具身智能重点实验室）；Shanghai Innovation Institute（上海创新研究院）；ETH Zürich（苏黎世联邦理工学院）
Bielefeld University（比勒费尔德大学）；Honda Research Institute Europe（本田欧洲研究院）
Research in social robotics is commonly focused on designing robots that imitate human behavior. While this might increase a user's satisfaction and acceptance of robots at first glance, it does not automatically aid a non-expert user in naturally interacting with robots, and might actually hurt their ability to correctly anticipate a robot's capabilities. We argue that a faulty mental model, that the user has of the robot, is one of the main sources of confusion. In this work we investigate how communicating technical concepts of robotic systems to users affects their mental models, and how t…
MIT Media Lab（麻省理工学院媒体实验室）；Pompeu Fabra University（庞培法布拉大学）
A significant number of college students suffer from mental health issues that impact their physical, social, and occupational outcomes. Various scalable technologies have been proposed in order to mitigate the negative impact of mental health disorders. However, the evaluation for these technologies, if done at all, often reports mixed results on improving users' mental health. We need to better understand the factors that align a user's attributes and needs with technology-based interventions for positive outcomes. In psychotherapy theory, therapeutic alliance and rapport between a therapist…
Model Reconciliation through Explainability and Collaborative Recovery in Assistive Robotics
German Aerospace Center (DLR), Robotics and Mechatronics Center (RMC)（德国航空航天中心（DLR）、机器人与机电中心）；University of Cambridge, Department of Engineering（剑桥大学工程系）；Universität Tübingen, Department of Psychology（图宾根大学心理学系）；Leibniz-Institut für Wissensmedien (IWM)（莱比锡知识媒体研究所）；Technical University of Munich（慕尼黑技术大学）
Personalized Mental State Evaluation in Human-Robot Interaction using Federated Learning
ARM-Lab University of Applied Sciences of Southern Switzerland (SUPSI)（ARM实验室 瑞士南方应用科学大学（SUPSI））；DAUIN Department Politecnico di Torino（DAUIN部门 托斯纳理工学院）；DIST Department Politecnico di Torino（DIST部门 托斯纳理工学院）
Evaluation of social robot navigation inherently requires human input due to its qualitative nature. Motivated by the need to scale human evaluation, we propose a general method for deploying interactive, rich-client robotic simulations on the web. Prior approaches implement specific web-compatible simulators or provide tools to build a simulator for a specific study. Instead, our approach builds on standard Linux tools to share a graphical desktop with remote users. We leverage these tools to deploy simulators on the web that would typically be constrained to desktop computing environments. A…
Heidelberg University Hospital（海德堡大学医院）；Karlsruhe Institute of Technology（卡尔斯鲁厄理工学院）
Cognitive cooperative assistance in robot-assisted surgery holds the potential to increase quality of care in minimally invasive interventions. Automation of surgical tasks promises to reduce the mental exertion and fatigue of surgeons. In this work, multi-agent reinforcement learning is demonstrated to be robust to the distribution shift introduced by pairing a learned policy with a human team member. Multi-agent policies are trained directly from images in simulation to control multiple instruments in a sub task of the minimally invasive removal of the gallbladder. These agents are evaluated…
We consider the design space available for social robots in terms of a hierarchy of functional definitions: the essential properties in terms of a locus of interaction, autonomy, intelligence, awareness of humans as possessors of mental state, and awareness of humans as social interactors. We also suggest that the emphasis on physical embodiment in some segments of the social robotics community has obscured commonalities with a class of agents that are identical in all other respects. These definitions naturally suggest research issues, directions, and possibilities which we explore. Social ro…
Robotics is the next frontier in the progress of Artificial Intelligence (AI), as the real world in which robots operate represents an enormous, complex, continuous state space with inherent real-time requirements. One extreme challenge in robotics is currently formed by autonomous drone racing. Human drone racers can fly through complex tracks at speeds of up to 190 km/h. Achieving similar speeds with autonomous drones signifies tackling fundamental problems in AI under extreme restrictions in terms of resources. In this article, we present the winning solution of the first AI Robotic Racing …
Okinawa Institute of Science and Technology Graduate University（冲绳科学技术大学院大学）
Human-robot interaction is becoming an interesting area of research in cognitive science, notably, for the study of social cognition. Interaction theorists consider primary intersubjectivity a non-mentalist, pre-theoretical, non-conceptual sort of processes that ground a certain level of communication and understanding, and provide support to higher-level cognitive skills. We argue this sort of low level cognitive interaction, where control is shared in dyadic encounters, is susceptible of study with neural robots. Hence, in this work we pursue three main objectives. Firstly, from the concept …
Learning mental states estimation through self-observation: a developmental synergy between intentions and beliefs representations in a deep-learning model of Theory of Mind
University of Essex（埃塞克斯大学）；Università Milano-Bicocca（米兰比可卡大学）；Istituto di Scienze e Tecnologie della Cognizione, Consiglio Nazionale della Ricerche（国家研究委员会认知科学与技术研究所）
Explainable deep learning improves human mental models of self-driving cars
Computer Science & Artificial Intelligence Laboratory (CSAIL), Massachusetts Institute of Technology（计算机科学与人工智能实验室（CSAIL），麻省理工学院）；Motional AD Inc.（Motional AD公司）；Department of Psychology and Center for Brain Science, Harvard University（心理学系和大脑科学中心，哈佛大学）；Department of Aeronautics and Astronautics, Massachusetts Institute of Technology（航空与宇航系，麻省理工学院）
University of Essex（埃塞克斯大学）；Università degli Studi di Milano Bicocca（米兰比可卡大学）
In complex environments, where the human sensory system reaches its limits, our behaviour is strongly driven by our beliefs about the state of the world around us. Accessing others' beliefs, intentions, or mental states in general, could thus allow for more effective social interactions in natural contexts. Yet these variables are not directly observable. Theory of Mind (TOM), the ability to attribute to other agents' beliefs, intentions, or mental states in general, is a crucial feature of human social interaction and has become of interest to the robotics community. Recently, new models that…
Ritsumeikan University（立命馆大学）；The University of Electro-Communications（电气通信大学）；Okayama Prefectural University（冈山县立大学）；Waseda University（早稻田大学）；National Institute of Advanced Industrial Science and Technology（产业技术综合研究所）
Humans can learn the use of language through physical interaction with their environment and semiotic communication with other people. It is very important to obtain a computational understanding of how humans can form a symbol system and obtain semiotic skills through their autonomous mental development. Recently, many studies have been conducted on the construction of robotic systems and machine-learning methods that can learn the use of language through embodied multimodal interaction with their environment and other systems. Understanding human social interactions and developing a robot th…
