Sensorimotor representation learning for an "active self" in robots: A model survey
University of Hamburg（汉堡大学）；Humboldt-Universität zu Berlin（柏林洪堡大学）；University of Potsdam（波茨坦大学）；Max Planck Institute for Human Cognitive and Brain Sciences（马克斯·普朗克人类认知与脑科学研究所）
Phuong D. H. Nguyen, Yasmin Kim Georgie, Ezgi Kayhan, Manfred Eppe, Verena Vanessa Hafner, Stefan Wermter
Safe human-robot interactions require robots to be able to learn how to behave appropriately in \sout{humans' world} \rev{spaces populated by people} and thus to cope with the challenges posed by our dynamic and unstructured environment, rather than being provided a rigid set of rules for operations. In humans, these capabilities are thought to be related to our ability to perceive our body in space, sensing the location of our limbs during movement, being aware of other objects and agents, and controlling our body parts to interact with them intentionally. Toward the next generation of robots with bio-inspired capacities, in this paper, we first review the developmental processes of underlying mechanisms of these abilities: The sensory representations of body schema, peripersonal space, and the active self in humans. Second, we provide a survey of robotics models of these sensory representations and robotics models of the self; and we compare these models with the human counterparts. Finally, we analyse what is missing from these robotics models and propose a theoretical computational framework, which aims to allow the emergence of the sense of self in artificial agents by developing sensory representations through self-exploration.
Modeling of physical human-robot collaborations is generally a challenging problem due to the unpredictive nature of human behavior. To address this issue, we present a data-efficient reinforcement learning framework which enables a robot to learn how to collaborate with a human partner. The robot learns the task from its own sensorimotor experiences in an unsupervised manner. The uncertainty of the human actions is modeled using Gaussian processes (GP) to implement action-value functions. Optimal action selection given the uncertain GP model is ensured by Bayesian optimization. We apply the f…
Robotic sensorimotor extensions (supernumerary limbs, prosthesis, handheld tools) are worn devices used to interact with the nearby environment, whether to assist the capabilities of impaired users or to enhance the dexterity of industrial operators. Despite numerous mechanical achievements, embedding these robotics devices remains critical due to their weight and discomfort. To emancipate from these mechanical constraints, we propose a new hybrid system using a virtually worn robotic arm in augmented-reality, and a real robotic manipulator servoed on such virtual representation. We aim at bri…
Frankfurt Institute of Advanced Studies（法兰克福高等研究院）
Open-ended learning is a core research field of developmental robotics and AI aiming to build learning machines and robots that can autonomously acquire knowledge and skills incrementally as infants and children. The first contribution of this work is to study the challenges posed by the previously proposed benchmark `REAL competition' aiming to foster the development of truly open-ended learning robot architectures. The competition involves a simulated camera-arm robot that: (a) in a first `intrinsic phase' acquires sensorimotor competence by autonomously interacting with objects; (b) in a se…
With the advances in robotic technology, research in human-robot collaboration (HRC) has gained in importance. For robots to interact with humans autonomously they need active decision making that takes human partners into account. However, state-of-the-art research in HRC does often assume a leader-follower division, in which one agent leads the interaction. We believe that this is caused by the lack of a reliable representation of the human and the environment to allow autonomous decision making. This problem can be overcome by an embodied approach to HRC which is inspired by psychological s…
Sensorimotor contingency theory offers a promising account of the nature of perception, a topic rarely addressed in the robotics community. We propose a developmental framework to address the problem of the autonomous acquisition of sensorimotor contingencies by a naive robot. While exploring the world, the robot internally encodes contingencies as predictive models that capture the structure they imply in its sensorimotor experience. Three preliminary applications are presented to illustrate our approach to the acquisition of perceptive abilities: discovering the environment, discovering obje…
Max Planck Institute for Intelligent Systems（马克斯·普朗克智能系统研究所）
Animals locomote robustly and agile, albeit significant sensorimotor delays of their nervous system. The sensorimotor control of legged robots is implemented with much higher frequencies-often in the kilohertz range-and sensor and actuator delays in the low millisecond range. But especially at harsh impacts with unknown touch-down timing, legged robots show unstable controller behaviors, while animals are seemingly not impacted. Here we examine this discrepancy and suggest a hybrid robotic leg and controller design. We implemented a physical, parallel joint compliance dimensioned in combinatio…
Research Institutes of Sweden（瑞典研究机构）；University of California, Berkeley（加州大学伯克利分校）；Luleå University of Technology（吕勒奥理工大学）
This correspondence comments on the findings reported in a recent Science Robotics article by Mitrokhin et al. [1]. The main goal of this commentary is to expand on some of the issues touched on in that article. Our experience is that hyperdimensional computing is very different from other approaches to computation and that it can take considerable exposure to its concepts before attaining practically useful understanding. Therefore, in order to provide an overview of the area to the first time reader of [1], the commentary includes a brief historic overview as well as connects the findings of…
Sensorimotor Attention and Language-based Regressions in Shared Latent Variables for Integrating Robot Motion Learning and LLM
Faculty of Science and Engineering, Waseda University（早稻田大学理工学部）；Artificial Intelligence Laboratories, Fujitsu Limited.（富士通株式会社人工智能研究所）；National Institute of Advanced Industrial Science and Technology（独立行政法人产业技术综合研究所）
In this work, we present the development of a neuro-inspired approach for characterizing sensorimotor relations in robotic systems. The proposed method has self-organizing and associative properties that enable it to autonomously obtain these relations without any prior knowledge of either the motor (e.g. mechanical structure) or perceptual (e.g. sensor calibration) models. Self-organizing topographic properties are used to build both sensory and motor maps, then the associative properties rule the stability and accuracy of the emerging connections between these maps. Compared to previous work…
On Model Adaptation for Sensorimotor Control of Robots
The Hong Kong Polytechnic University, Hung Hom, Kowloon, Hong Kong（香港理工大学）；University of Montpellier / LIRMM, Montpellier, France（蒙彼利埃大学 / LIRMM）；The Chinese University of Hong Kong, Shatin, New Territories, Hong Kong（香港中文大学）
MPI for Mathematics in the Sciences（马克斯·普朗克数学科学研究所）；Santa Fe Institute（圣塔菲研究所）
This work presents a novel learning method in the context of embodied artificial intelligence and self-organization, which has as few assumptions and restrictions as possible about the world and the underlying model. The learning rule is derived from the principle of maximizing the predictive information in the sensorimotor loop. It is evaluated on robot chains of varying length with individually controlled, non-communicating segments. The comparison of the results shows that maximizing the predictive information per wheel leads to a higher coordinated behavior of the physically connected robo…
University of Bristol（布里斯托大学）；Universit di Pisa（比萨大学）；Istituto Italiano di Tecnologia（意大利理工学院）
In this work, we report on the integrated sensorimotor control of the Pisa/IIT SoftHand, an anthropomorphic soft robot hand designed around the principle of adaptive synergies, with the BRL tactile fingertip (TacTip), a soft biomimetic optical tactile sensor based on the human sense of touch. Our focus is how a sense of touch can be used to control an anthropomorphic hand with one degree of actuation, based on an integration that respects the hand's mechanical functionality. We consider: (i) closed-loop tactile control to establish a light contact on an unknown held object, based on the struct…
AI Lab, SoftBank Robotics Europe（软银机器人欧洲公司人工智能实验室）
Robots act in their environment through sequences of continuous motor commands. Because of the dimensionality of the motor space, as well as the infinite possible combinations of successive motor commands, agents need compact representations that capture the structure of the resulting displacements. In the case of an autonomous agent with no a priori knowledge about its sensorimotor apparatus, this compression has to be learned. We propose to use Recurrent Neural Networks to encode motor sequences into a compact representation, which is used to predict the consequence of motor sequences in ter…
Bio-inspired sensorimotor control systems may be appealing to roboticists who try to solve problems of multiDOF humanoids and human-robot interactions. This paper presents a simple posture control concept from neuroscience, called disturbance estimation and compensation, DEC concept [1]. It provides human-like mechanical compliance due to low loop gain, tolerance of time delays, and automatic adjustment to changes in external disturbance scenarios. Its outstanding feature is that it uses feedback of multisensory disturbance estimates rather than 'raw' sensory signals for disturbance compensati…
Unsupervised Discovery of Behavioral Primitives from Sensorimotor Dynamic Functional Connectivity
Munich Institute of Robotics and Machine Intelligence, Technical University of Munich（慕尼黑机器人与机器智能研究所，慕尼黑技术大学）；Faculty of Electrical Engineering, Czech Technical University in Prague（布拉格捷克技术大学电子工程学院）
In a developmental framework, autonomous robots need to explore the world and learn how to interact with it. Without an a priori model of the system, this opens the challenging problem of having robots master their interface with the world: how to perceive their environment using their sensors, and how to act in it using their motors. The sensorimotor approach of perception claims that a naive agent can learn to master this interface by capturing regularities in the way its actions transform its sensory inputs. In this paper, we apply such an approach to the discovery and mastery of the visual…
Institute for Cognitive Systems（认知系统研究所）；Technische Universität München（慕尼黑工业大学）
The predictive functions that permit humans to infer their body state by sensorimotor integration are critical to perform safe interaction in complex environments. These functions are adaptive and robust to non-linear actuators and noisy sensory information. This paper introduces a computational perceptual model based on predictive processing that enables any multisensory robot to learn, infer and update its body configuration when using arbitrary sensors with Gaussian additive noise. The proposed method integrates different sources of information (tactile, visual and proprioceptive) to drive …
The functional and temporal roles of gaze evolve across the phases and constraints of multi-stage robot-mediated manipulation
The BioRobotics Institute, Scuola Superiore Sant’Anna（生物机器人研究所，高等圣安娜学院）；Department of Excellence in Robotics and AI, Scuola Superiore Sant’Anna（机器人与人工智能卓越部门，高等圣安娜学院）
KTH Royal Institute of Technology（瑞典皇家理工学院）；Aalto University（阿尔托大学）；Alanya Alaaddin Keykubat University（阿拉尼亚·阿拉丁·库库巴特大学）
To coordinate actions with an interaction partner requires a constant exchange of sensorimotor signals. Humans acquire these skills in infancy and early childhood mostly by imitation learning and active engagement with a skilled partner. They require the ability to predict and adapt to one's partner during an interaction. In this work we want to explore these ideas in a human-robot interaction setting in which a robot is required to learn interactive tasks from a combination of observational and kinesthetic learning. To this end, we propose a deep learning framework consisting of a number of c…
AI Lab of SoftBank Robotics Europe (Aldebaran)（软银机器人欧洲公司人工智能实验室（Aldebaran））
Autonomous robots need to be able to adapt to unforeseen situations and to acquire new skills through trial and error. Reinforcement learning in principle offers a suitable methodological framework for this kind of autonomous learning. However current computational reinforcement learning agents mostly learn each individual skill entirely from scratch. How can we enable artificial agents, such as robots, to acquire some form of generic knowledge, which they could leverage for the learning of new skills? This paper argues that, like the brain, the cognitive system of artificial agents has to dev…
Babes-Bolyai University（巴比什-波雅依大学）；Goethe University Frankfurt（法兰克福歌德大学）
Self-organized robots may develop attracting states within the sensorimotor loop, that is within the phase space of neural activity, body, and environmental variables. Fixpoints, limit cycles, and chaotic attractors correspond in this setting to a non-moving robot, to directed, and to irregular locomotion respectively. Short higher-order control commands may hence be used to kick the system from one self-organized attractor robustly into the basin of attraction of a different attractor, a concept termed here as kick control. The individual sensorimotor states serve in this context as highly co…
AI Lab, SoftBank Robotics Europe（AI实验室，软银机器人欧洲公司（SoftBank Robotics Europe）AI实验室（注：此处“AI Lab”作为机构内部部门，整体机构为“SoftBank Robotics Europe”，按要求提取到公司级别））
In Machine Learning and Robotics, the semantic content of visual features is usually provided to the system by a human who interprets its content. On the contrary, strictly unsupervised approaches have difficulties relating the statistics of sensory inputs to their semantic content without also relying on prior knowledge introduced in the system. We proposed in this paper to tackle this problem from a sensorimotor perspective. In line with the Sensorimotor Contingencies Theory, we make the fundamental assumption that the semantic content of sensory inputs at least partially stems from the way …
Despite its omnipresence in robotics application, the nature of spatial knowledge and the mechanisms that underlie its emergence in autonomous agents are still poorly understood. Recent theoretical works suggest that the Euclidean structure of space induces invariants in an agent's raw sensorimotor experience. We hypothesize that capturing these invariants is beneficial for sensorimotor prediction and that, under certain exploratory conditions, a motor representation capturing the structure of the external space should emerge as a byproduct of learning to predict future sensory experiences. We…
During visuomotor tasks, robots must compensate for temporal delays inherent in their sensorimotor processing systems. Delay compensation becomes crucial in a dynamic environment where the visual input is constantly changing, e.g., during the interacting with a human demonstrator. For this purpose, the robot must be equipped with a prediction mechanism for using the acquired perceptual experience to estimate possible future motor commands. In this paper, we present a novel neural network architecture that learns prototypical visuomotor representations and provides reliable predictions on the b…
Sorbonne Universités, UPMC Univ Paris 06（索邦大学，巴黎第六大学）；CNRS（法国国家科学研究中心）；Université Paris Descartes（巴黎笛卡尔大学）；École Normale Supérieure（巴黎高等师范学院）
The design of robotic systems is largely dictated by our purely human intuition about how we perceive the world. This intuition has been proven incorrect with regard to a number of critical issues, such as visual change blindness. In order to develop truly autonomous robots, we must step away from this intuition and let robotic agents develop their own way of perceiving. The robot should start from scratch and gradually develop perceptual notions, under no prior assumptions, exclusively by looking into its sensorimotor experience and identifying repetitive patterns and invariants. One of the m…
The current dominant paradigm in sensorimotor control, whether imitation or reinforcement learning, is to train policies directly in raw action spaces such as torque, joint angle, or end-effector position. This forces the agent to make decisions individually at each timestep in training, and hence, limits the scalability to continuous, high-dimensional, and long-horizon tasks. In contrast, research in classical robotics has, for a long time, exploited dynamical systems as a policy representation to learn robot behaviors via demonstrations. These techniques, however, lack the flexibility and ge…
We investigate the sensorimotor loop of simple robots simulated within the LPZRobots environment from the point of view of dynamical systems theory. For a robot with a cylindrical shaped body and an actuator controlled by a single proprioceptual neuron we find various types of periodic motions in terms of stable limit cycles. These are self-organized in the sense, that the dynamics of the actuator kicks in only, for a certain range of parameters, when the barrel is already rolling, stopping otherwise. The stability of the resulting rolling motions terminates generally, as a function of the con…
SoftBank Robotics EU（软银机器人欧洲）；Univ. Paris 05 Descartes（巴黎第五大学笛卡尔）；CNRS（法国国家科学研究中心）；Univ. Paris 06 UPMC（巴黎第六大学皮埃尔和玛丽居里大学）
In line with the sensorimotor contingency theory, we investigate the problem of the perception of space from a fundamental sensorimotor perspective. Despite its pervasive nature in our perception of the world, the origin of the concept of space remains largely mysterious. For example in the context of artificial perception, this issue is usually circumvented by having engineers pre-define the spatial structure of the problem the agent has to face. We here show that the structure of space can be autonomously discovered by a naive agent in the form of sensorimotor regularities, that correspond t…
We present a general framework to autonomously achieve a task, where autonomy is acquired by learning sensorimotor patterns of a robot, while it is interacting with its environment. To accomplish the task, using the learned sensorimotor contingencies, our approach predicts a sequence of actions that will lead to the desirable observations. Gaussian processes (GP) with automatic relevance determination is used to learn the sensorimotor mapping. In this way, relevant sensory and motor components can be systematically found in high-dimensional sensory and motor spaces. We propose an incremental G…
Stanford University（斯坦福大学）；X, The Moonshot Factory（X，登月工厂）
A key challenge towards the goal of multi-part assembly tasks is finding robust sensorimotor control methods in the presence of uncertainty. In contrast to previous works that rely on a priori knowledge on whether two parts match, we aim to learn this through physical interaction. We propose a hierarchical approach that enables a robot to autonomously assemble parts while being uncertain about part types and positions. In particular, our probabilistic approach learns a set of differentiable filters that leverage the tactile sensorimotor trace from failed assembly attempts to update its belief …
University of Birmingham（伯明翰大学）；Waseda University（早稻田大学）
The hype about sensorimotor learning is currently reaching high fever, thanks to the latest advancement in deep learning. In this paper, we present an open-source framework for collecting large-scale, time-synchronised synthetic data from highly disparate sensory modalities, such as audio, video, and proprioception, for learning robot manipulation tasks. We demonstrate the learning of non-linear sensorimotor mappings for a humanoid drumming robot that generates novel motion sequences from desired audio data using cross-modal correspondences. We evaluate our system through the quality of its cr…
Instituto Superior Técnico, Universidade de Lisboa（里斯本大学高等理工学院）；Champalimaud Centre for the Unknown（尚帕利莫未知中心）；Queen Mary University of London（伦敦玛丽女王大学）
One of the open challenges in designing robots that operate successfully in the unpredictable human environment is how to make them able to predict what actions they can perform on objects, and what their effects will be, i.e., the ability to perceive object affordances. Since modeling all the possible world interactions is unfeasible, learning from experience is required, posing the challenge of collecting a large amount of experiences (i.e., training data). Typically, a manipulative robot operates on external objects by using its own hands (or similar end-effectors), but in some cases the us…
Motor babbling and goal babbling has been used for sensorimotor learning of highly redundant systems in soft robotics. Recent works in goal babbling has demonstrated successful learning of inverse kinematics (IK) on such systems, and suggests that babbling in the goal space better resolves motor redundancy by learning as few sensorimotor mapping as possible. However, for musculoskeletal robot systems, motor redundancy can be of useful information to explain muscle activation patterns, thus the term motor abundance. In this work, we introduce some simple heuristics to empirically define the unk…
Max Planck Institute for Mathematics in the Science（马克斯·普朗克科学数学研究所）；IST Austria（奥地利科学技术研究所）
Grounding autonomous behavior in the nervous system is a fundamental challenge for neuroscience. In particular, the self-organized behavioral development provides more questions than answers. Are there special functional units for curiosity, motivation, and creativity? This paper argues that these features can be grounded in synaptic plasticity itself, without requiring any higher level constructs. We propose differential extrinsic plasticity (DEP) as a new synaptic rule for self-learning systems and apply it to a number of complex robotic systems as a test case. Without specifying any purpose…
Similar to humans, robots benefit from interacting with their environment through a number of different sensor modalities, such as vision, touch, sound. However, learning from different sensor modalities is difficult, because the learning model must be able to handle diverse types of signals, and learn a coherent representation even when parts of the sensor inputs are missing. In this paper, a multimodal variational autoencoder is proposed to enable an iCub humanoid robot to learn representations of its sensorimotor capabilities from different sensor modalities. The proposed model is able to (…
SoftBank Robotics Europe（软银机器人欧洲公司）；Humboldt-Universität zu Berlin（柏林洪堡大学）
This work investigates how a naive agent can acquire its own body image in a self-supervised way, based on the predictability of its sensorimotor experience. Our working hypothesis is that, due to its temporal stability, an agent's body produces more consistent sensory experiences than the environment, which exhibits a greater variability. Given its motor experience, an agent can thus reliably predict what appearance its body should have. This intrinsic predictability can be used to automatically isolate the body image from the rest of the environment. We propose a two-branches deconvolutional…
