As the current trend of artificial intelligence is shifting towards self-supervised learning, conventional norms such as highly curated domain-specific data, application-specific learning models, extrinsic reward based learning policies etc. might not provide with the suitable ground for such developments. In this paper, we introduce SEDRo, a Simulated Environment for Developmental Robotics which allows a learning agent to have similar experiences that a human infant goes through from the fetus stage up to 12 months. A series of simulated tests based on developmental psychology will be used to evaluate the progress of a learning model.
Even with impressive advances in application-specific models, we still lack knowledge about how to build a model that can learn in a human-like way and do multiple tasks. To learn in a human-like way, we need to provide a diverse experience that is comparable to humans. In this paper, we introduce our ongoing effort to build a simulated environment for developmental robotics (SEDRo). SEDRo provides diverse human experiences ranging from those of a fetus to a 12th-month-old. A series of simulated tests based on developmental psychology will be used to evaluate the progress of a learning model. …
The agent program, called Samu, is an experiment to build a disembodied DevRob (Developmental Robotics) chatter bot that can talk in a natural language like humans do. One of the main design feature is that Samu can be interacted with using only a character terminal. This is important not only for practical aspects of Turing test or Loebner prize, but also for the study of basic principles of Developmental Robotics. Our purpose is to create a rapid prototype of Q-learning with neural network approximators for Samu. We sketch out the early stages of the development process of this prototype, wh…
Task-specific AI agents are showing remarkable performance across different domains. But modeling generalized AI agents like human intelligence will require more than current datasets or only reward-based environments that don't include experiences that an infant gathers throughout its initial stages. In this paper, we present Simulated Environment for Developmental Robotics (SEDRo). It simulates the environments for a baby agent that a human baby experiences throughout the pre-born fetus stage to post-birth 12 months. SEDRo also includes a mother character to provide social interaction with t…
Despite the recent advancement in the social robotic field, important limitations restrain its progress and delay the application of robots in everyday scenarios. In the present paper, we propose to develop computational models inspired by our knowledge of human infants' social adaptive abilities. We believe this may provide solutions at an architectural level to overcome the limits of current systems. Specifically, we present the functional advantages that adaptive Theory of Mind (ToM) systems would support in robotics (i.e., mentalizing for belief understanding, proactivity and preparation, …
Autonomous lifelong development and learning is a fundamental capability of humans, differentiating them from current deep learning systems. However, other branches of artificial intelligence have designed crucial ingredients towards autonomous learning: curiosity and intrinsic motivation, social learning and natural interaction with peers, and embodiment. These mechanisms guide exploration and autonomous choice of goals, and integrating them with deep learning opens stimulating perspectives. Deep learning (DL) approaches made great advances in artificial intelligence, but are still far away f…
Learning mental states estimation through self-observation: a developmental synergy between intentions and beliefs representations in a deep-learning model of Theory of Mind
University of Essex（埃塞克斯大学）；Università Milano-Bicocca（米兰比可卡大学）；Istituto di Scienze e Tecnologie della Cognizione, Consiglio Nazionale della Ricerche（国家研究委员会认知科学与技术研究所）
Ritsumeikan University（立命馆大学）；Keio University（庆应义塾大学）；The University of Tokyo（东京大学）；Università Milano-Bicocca（米兰比可卡大学）；University of Essex（埃塞克斯大学）；Donders Institute for Brain, Cognition and Behaviour（唐德斯脑、认知与行为研究所）；Cajal International Neuroscience Center, Spanish National Research Council（西班牙国家研究委员会卡哈尔国际神经科学中心）；Bogazici University（博阿齐奇大学）；Queen Mary University of London（伦敦玛丽女王大学）；The University of Electro-Communications（电气通信大学）；National Autonomous University of Mexico（墨西哥国立自治大学）；Universidad Autónoma del Estado de Morelos（莫雷洛斯州自治大学）；Institute of Cognitive Sciences and Technologies, National Research Council of Italy（意大利国家研究委员会认知科学与技术研究所）
Creating autonomous robots that can actively explore the environment, acquire knowledge and learn skills continuously is the ultimate achievement envisioned in cognitive and developmental robotics. Their learning processes should be based on interactions with their physical and social world in the manner of human learning and cognitive development. Based on this context, in this paper, we focus on the two concepts of world models and predictive coding. Recently, world models have attracted renewed attention as a topic of considerable interest in artificial intelligence. Cognitive systems learn…
University of Hamburg（汉堡大学）；Humboldt-Universität zu Berlin（柏林洪堡大学）；University of Potsdam（波茨坦大学）；Max Planck Institute for Human Cognitive and Brain Sciences（马克斯·普朗克人类认知与脑科学研究所）
Safe human-robot interactions require robots to be able to learn how to behave appropriately in \sout{humans' world} \rev{spaces populated by people} and thus to cope with the challenges posed by our dynamic and unstructured environment, rather than being provided a rigid set of rules for operations. In humans, these capabilities are thought to be related to our ability to perceive our body in space, sensing the location of our limbs during movement, being aware of other objects and agents, and controlling our body parts to interact with them intentionally. Toward the next generation of robots…
LIRIS, CNRS UMR 5205 Ecole Centrale de Lyon（里昂中央理工学院 LIRIS 实验室（法国国家科学研究中心 UMR 5205））
In robotics, methods and softwares usually require optimizations of hyperparameters in order to be efficient for specific tasks, for instance industrial bin-picking from homogeneous heaps of different objects. We present a developmental framework based on long-term memory and reasoning modules (Bayesian Optimisation, visual similarity and parameters bounds reduction) allowing a robot to use meta-learning mechanism increasing the efficiency of such continuous and constrained parameters optimizations. The new optimization, viewed as a learning for the robot, can take advantage of past experience…
Ritsumeikan University（立命馆大学）；The University of Tokyo（东京大学）；The Whole Brain Architecture Initiative（全脑架构倡议）；Osaka University（大阪大学）；Okinawa Institute of Science and Technology Graduate University（冲绳科学技术大学院大学）；Tamagawa University（玉川大学）；The University of Electro-Communications（电气通信大学）
Building a humanlike integrative artificial cognitive system, that is, an artificial general intelligence (AGI), is the holy grail of the artificial intelligence (AI) field. Furthermore, a computational model that enables an artificial system to achieve cognitive development will be an excellent reference for brain and cognitive science. This paper describes an approach to develop a cognitive architecture by integrating elemental cognitive modules to enable the training of the modules as a whole. This approach is based on two ideas: (1) brain-inspired AI, learning human brain architecture to b…
Design and development of opto-neural processors for simulation of neural networks trained in image detection for potential implementation in hybrid robotics
University of Cincinnati（辛辛那提大学）；Visvesvaraya Technological University（维斯韦斯瓦拉亚理工大学）
A3RNN: Bi-directional Fusion of Bottom-up and Top-down Process for Developmental Visual Attention in Robots
Department of Intermedia Art and Science, Waseda University（媒体艺术与科学系，早稻田大学）；Research Development Group, Hitachi, Ltd.（日立株式会社研发发展组）；National Institute of Advanced Industrial Science and Technology (AIST)（国家先进工业科学与技术研究院）
CNRS（法国国家科学研究中心）；ISIR（智能系统与机器人研究所）；U2IS（U2IS实验室）；INRIA Flowers（法国国家信息与自动化研究所Flowers团队）；ENSTA（法国国立高等先进技术学院）；Institut Polytechnique Paris（巴黎理工学院）；University of Edinburgh（爱丁堡大学）
Robots are still limited to controlled conditions, that the robot designer knows with enough details to endow the robot with the appropriate models or behaviors. Learning algorithms add some flexibility with the ability to discover the appropriate behavior given either some demonstrations or a reward to guide its exploration with a reinforcement learning algorithm. Reinforcement learning algorithms rely on the definition of state and action spaces that define reachable behaviors. Their adaptation capability critically depends on the representations of these spaces: small and discrete spaces re…
What are the functions of curiosity? What are the mechanisms of curiosity-driven learning? We approach these questions about the living using concepts and tools from machine learning and developmental robotics. We argue that curiosity-driven learning enables organisms to make discoveries to solve complex problems with rare or deceptive rewards. By fostering exploration and discovery of a diversity of behavioural skills, and ignoring these rewards, curiosity can be efficient to bootstrap learning when there is no information, or deceptive information, about local improvement towards these probl…
This paper outlines a perspective on the future of AI, discussing directions for machines models of human-like intelligence. We explain how developmental and evolutionary theories of human cognition should further inform artificial intelligence. We emphasize the role of ecological niches in sculpting intelligent behavior, and in particular that human intelligence was fundamentally shaped to adapt to a constantly changing socio-cultural environment. We argue that a major limit of current work in AI is that it is missing this perspective, both theoretically and experimentally. Finally, we discus…
Donders Institute for Brain, Cognition, and Behaviour（唐德斯大脑、认知与行为研究所）；Radboud University（拉德堡德大学）；Joint Research Centre, European Commission（欧盟委员会联合研究中心）；Ghent University（根特大学）；imec（微电子研究中心）；Instituto Superio Técnico（高等技术学院）；University of Lisbon（里斯本大学）
Understanding and modelling children's cognitive processes and their behaviour in the context of their interaction with robots and social artificial intelligence systems is a fundamental prerequisite for meaningful and effective robot interventions. However, children's development involve complex faculties such as exploration, creativity and curiosity which are challenging to model. Also, often children express themselves in a playful way which is different from a typical adult behaviour. Different children also have different needs, and it remains a challenge in the current state of the art t…
We want to build robots that are useful in unstructured real world applications, such as doing work in the household. Grasping in particular is an important skill in this domain, yet it remains a challenge. One of the key hurdles is handling unexpected changes or motion in the objects being grasped and kinematic noise or other errors in the robot. This paper proposes an approach to learning a closed-loop controller for robotic grasping that dynamically guides the gripper to the object. We use a wrist-mounted sensor to acquire depth images in front of the gripper and train a convolutional neura…
University of Tennessee（田纳西大学）；Miami University（迈阿密大学）；University of Tennessee Health Science Center（田纳西大学健康科学中心）；University of Iowa（爱荷华大学）
People with Alzheimer's disease and related dementias (ADRD) often show the problem of repetitive questioning, which brings a great burden on persons with ADRD (PwDs) and their caregivers. Conversational robots hold promise of coping with this problem and hence alleviating the burdens on caregivers. In this paper, we proposed a partially observable markov decision process (POMDP) model for the PwD-robot interaction in the context of repetitive questioning, and used Q-learning to learn an adaptive conversation strategy (i.e., rate of follow-up question and difficulty of follow-up question) towa…
Mastering robotic manipulation skills through reinforcement learning (RL) typically requires the design of shaped reward functions. Recent developments in this area have demonstrated that using sparse rewards, i.e. rewarding the agent only when the task has been successfully completed, can lead to better policies. However, state-action space exploration is more difficult in this case. Recent RL approaches to learning with sparse rewards have leveraged high-quality human demonstrations for the task, but these can be costly, time consuming or even impossible to obtain. In this paper, we propose …
Learning Low-Level Causal Relations using a Simulated Robotic Arm
Comenius University Bratislava（布拉迪斯拉发考门斯基大学）；Faculty of Mathematics, Physics and Informatics, Comenius University Bratislava（布拉迪斯拉发考门斯基大学数学、物理与信息学院）；University of Hamburg（汉堡大学）
Inria（法国国家信息与自动化研究所）；CNRS（法国国家科学研究中心）；Université de Lorraine（洛林大学）
Meta-learning algorithms can accelerate the model-based reinforcement learning (MBRL) algorithms by finding an initial set of parameters for the dynamical model such that the model can be trained to match the actual dynamics of the system with only a few data-points. However, in the real world, a robot might encounter any situation starting from motor failures to finding itself in a rocky terrain where the dynamics of the robot can be significantly different from one another. In this paper, first, we show that when meta-training situations (the prior situations) have such diverse dynamics, usi…
This paper proposes a detailed and extensive comparison of the Trust Region Policy Optimization and DeepQ-Network with Normalized Advantage Functions with respect to other state of the art algorithms, namely Deep Deterministic Policy Gradient and Vanilla Policy Gradient. Comparisons demonstrate that the former have better performances then the latter when asking robotic arms to accomplish manipulation tasks such as reaching a random target pose and pick &placing an object. Both simulated and real-world experiments are provided. Simulation lets us show the procedures that we adopted to precisel…
Model-free deep reinforcement learning has been shown to exhibit good performance in domains ranging from video games to simulated robotic manipulation and locomotion. However, model-free methods are known to perform poorly when the interaction time with the environment is limited, as is the case for most real-world robotic tasks. In this paper, we study how maximum entropy policies trained using soft Q-learning can be applied to real-world robotic manipulation. The application of this method to real-world manipulation is facilitated by two important features of soft Q-learning. First, soft Q-…
Instrumenting and collecting annotated visual grasping datasets to train modern machine learning algorithms can be extremely time-consuming and expensive. An appealing alternative is to use off-the-shelf simulators to render synthetic data for which ground-truth annotations are generated automatically. Unfortunately, models trained purely on simulated data often fail to generalize to the real world. We study how randomized simulated environments and domain adaptation methods can be extended to train a grasping system to grasp novel objects from raw monocular RGB images. We extensively evaluate…
Monash University（莫纳什大学）；Defence Science and Technology Group（国防科学技术集团）；Sensor Analytics（传感器分析公司）
This paper explores the use of a novel form of Hierarchical Graph Neurons (HGN) for in-operation behaviour selection in a swarm of robotic agents. This new HGN is called Robotic-HGN (R-HGN), as it matches robot environment observations to environment labels via fusion of match probabilities from both temporal and intra-swarm collections. This approach is novel for HGN as it addresses robotic observations being pseudo-continuous numbers, rather than categorical values. Additionally, the proposed approach is memory and computation-power conservative and thus is acceptable for use in mobile devic…
Our effort is toward unifying GAN and DRL algorithms into a unifying AI model (AGI or general-purpose AI or artificial general intelligence which has general-purpose applications to: (A) offline learning (of stored data) like GAN in (un/semi-/fully-)SL setting such as big data analytics (mining) and visualization; (B) online learning (of real or simulated devices) like DRL in RL setting (with/out environment reward) such as (real or simulated) robotics and control; Our core proposal is adding an (generative/predictive) environment model to the actor-critic (model-free) architecture which resul…
In a developmental framework, autonomous robots need to explore the world and learn how to interact with it. Without an a priori model of the system, this opens the challenging problem of having robots master their interface with the world: how to perceive their environment using their sensors, and how to act in it using their motors. The sensorimotor approach of perception claims that a naive agent can learn to master this interface by capturing regularities in the way its actions transform its sensory inputs. In this paper, we apply such an approach to the discovery and mastery of the visual…
Success stories of applied machine learning can be traced back to the datasets and environments that were put forward as challenges for the community. The challenge that the community sets as a benchmark is usually the challenge that the community eventually solves. The ultimate challenge of reinforcement learning research is to train real agents to operate in the real environment, but until now there has not been a common real-world RL benchmark. In this work, we present a prototype real-world environment from OffWorld Gym -- a collection of real-world environments for reinforcement learning …
Applying Deep Reinforcement Learning (DRL) to complex tasks in the field of robotics has proven to be very successful in the recent years. However, most of the publications focus either on applying it to a task in simulation or to a task in a real world setup. Although there are great examples of combining the two worlds with the help of transfer learning, it often requires a lot of additional work and fine-tuning to make the setup work effectively. In order to increase the use of DRL with real robots and reduce the gap between simulation and real world robotics, we propose an open source tool…
