Imitation plays an important role in development, being one of the precursors of social cognition. Even though some children with autism imitate spontaneously and other children with autism can learn to imitate, the dynamics of imitation is affected in the large majority of cases. Existing studies from the literature suggest that robots can be used to teach children with autism basic interaction skills like imitation. Based on these findings, in this study, we investigate if children with autism show more social engagement when interacting with an imitative robot (Fig 1) compared to a human partner in a motor imitation task.
In this exploratory study we assessed how attitudes of children with autism spectrum disorder (ASD) towards robots together with children's autism-related social impairments are linked to indicators of children's preference of an interaction with a robot over an interaction with a person. We found that children with ASD have overall positive attitudes towards robots and that they often prefer interacting with a robot than with a person. Several of children's attitudes were linked to children's longer gazes towards a robot compared to a person. Autism-related social impairments were linked to m…
There is a lack of autonomy on traditional Robot-Assisted Therapy systems interacting with children with autism. To overcome this limitation a supervised autonomous robot controller is being built. In this paper we present a multilayer reactive system within such controller. The goal of this Reactive system is to allow the robot to appropriately react to the child's behavior creating the illusion of being alive.
University of Tehran（德黑兰大学）；Intelligent Systems Research Institute, SKKU（成均馆大学智能系统研究所）；Allameh Tabatabái University（阿拉梅塔巴塔巴伊大学）；Iran University of Medical Sciences（伊朗医科大学）；Shahid Beheshti University（沙希德贝赫什提大学）
Robot Assisted Therapy is a new paradigm in many therapies such as the therapy of children with autism spectrum disorder. In this paper we present the use of a parrot-like robot as an assistive tool in turn taking therapy. The therapy is designed in the form of a card game between a child with autism and a therapist or the robot. The intervention was implemented in a single subject study format and the effect sizes for different turn taking variables are calculated. The results show that the child robot interaction had larger effect size than the child trainer effect size in most of the turn t…
Affect-aware socially assistive robotics (SAR) tutors have great potential to augment and democratize professional therapeutic interventions for children with autism spectrum disorders (ASD) from different socioeconomic backgrounds. However, the majority of research on SAR for ASD has been on teaching cognitive and/or social skills, not on addressing users' emotional needs for real-world social situations. To bridge that gap, this work aims to develop personalized affect-aware SAR tutors to help alleviate social anxiety and foster social grit-the growth mindset for social skill development-in …
Neurofeedback games are an effective and playful approach to enhance certain social and attentional capabilities in children with autism, which are promising to become widely accessible along with the commercialization of mobile EEG modules. However, little industry-based experiences are shared, regarding how to better design neurofeedback games to fine-tune their playability and user experiences for autistic children. In this paper, we review the experiences we gained from industry practice, in which a series of mobile EEG neurofeedback games have been developed for preschool autistic childre…
University of Southern California（南加州大学）；Massachusetts Institute of Technology（麻省理工学院）
Affect-aware socially assistive robotics (SAR) has shown great potential for augmenting interventions for children with autism spectrum disorders (ASD). However, current SAR cannot yet perceive the unique and diverse set of atypical cognitive-affective behaviors from children with ASD in an automatic and personalized fashion in long-term (multi-session) real-world interactions. To bridge this gap, this work designed and validated personalized models of arousal and valence for children with ASD using a multi-session in-home dataset of SAR interventions. By training machine learning (ML) algorit…
National University of Singapore（新加坡国立大学）；Ministry of Education, Singapore（新加坡教育部）
This paper reviews the use of socially interactive robots to assist in the therapy of children with autism. The extent to which the robots were successful in helping the children in their social, emotional, and communication deficits was investigated. Child-robot interactions were scrutinized with respect to the different target behaviors that are to be elicited from a child during therapy. These behaviors were thoroughly examined with respect to a childs development needs. Most importantly, experimental data from the surveyed works were extracted and analyzed in terms of the target behaviors …
A good teaching method is incomprehensible for an autistic child. The autism spectrum disorder is a very diverse phenomenon. It is said that no two autistic children are the same. So, something that works for one child may not be fit for another. The same case is true for their education. Different children need to be approached with different teaching methods. But it is quite hard to identify the appropriate teaching method. As the term itself explains, the autism spectrum disorder is like a spectrum. There are multiple factors to determine the type of autism of a child. A child might even be…
Social visual behavior, as a type of non-verbal communication, plays a central role in studying social cognitive processes in interactive and complex settings of autism therapy interventions. However, for social visual behavior analytics in children with autism, it is challenging to collect gaze data manually and evaluate them because it costs a lot of time and effort for human coders. In this paper, we introduce a social visual behavior analytics approach by quantifying the mutual gaze performance of children receiving play-based autism interventions using an automated mutual gaze detection f…
Visual Programming Tools (VPTs) allow users to create interactive media projects such as games and animations using visual representations of programming concepts. Although VPTs have been shown to have huge potential for teaching children with cognitive impairments including those with Autism Spectrum Condition (ASC), research has shown that existing VPTs may not be accessible to them. Therefore, this study proposes a set of recommendations for the design of accessible VPTs for children with ASC. Recommendations were initially gathered and validated by interviewing experts (n=7). The interview…
This research addressed whether interactions with social robots could facilitate joint attention of the autism spectrum disorder (ASD). Two conditions of initiators, namely 'Human' vs. 'Robot' were measured with 15 children with ASD and 15 age-matched typically developing (TD) children. Apart from fixation and gaze transition, a new longest common subsequence (LCS) approach was proposed to analyze eye-movement traces. Results revealed that children with ASD showed deficits of joint attention. Compared to the human agent, robot facilitate less fixations towards the targets, but it attracted mor…
With most recent estimates giving an incidence rate of 1 in 68 children in the United States, the autism spectrum disorder (ASD) is a growing public health crisis. Many of these children struggle to make eye contact, recognize facial expressions, and engage in social interactions. Today the standard for treatment of the core autism-related deficits focuses on a form of behavior training known as Applied Behavioral Analysis. To address perceived deficits in expression recognition, ABA approaches routinely involve the use of prompts such as flash cards for repetitive emotion recognition training…
Socially assistive robotics (SAR) has great potential to provide accessible, affordable, and personalized therapeutic interventions for children with autism spectrum disorders (ASD). However, human-robot interaction (HRI) methods are still limited in their ability to autonomously recognize and respond to behavioral cues, especially in atypical users and everyday settings. This work applies supervised machine learning algorithms to model user engagement in the context of long-term, in-home SAR interventions for children with ASD. Specifically, we present two types of engagement models for each …
Autism spectrum disorder (ASD) is a developmental disorder that influences the communication and social behavior of a person in a way that those in the spectrum have difficulty in perceiving other people's facial expressions, as well as presenting and communicating emotions and affect via their own faces and bodies. Some efforts have been made to predict and improve children with ASD's affect states in play therapy, a common method to improve children's social skills via play and games. However, many previous works only used pre-trained models on benchmark emotion datasets and failed to consid…
Children with Autism Spectrum Disorder (ASD) experience deficits in verbal and nonverbal communication skills including motor control, turn-taking, and emotion recognition. Innovative technology, such as socially assistive robots, has shown to be a viable method for Autism therapy. This paper presents a novel robot-based music-therapy platform for modeling and improving the social responses and behaviors of children with ASD. Our autonomous social interactive system consists of three modules. We adopted Short-time Fourier Transform and Levenshtein distance to fulfill the design requirements: a…
Technische Universität München（慕尼黑工业大学）；University of Cambridge（剑桥大学）
Individuals with Autism Spectrum Conditions (ASC) have marked difficulties using verbal and non-verbal communication for social interaction. The running ASC-Inclusion project aims to help children with ASC by allowing them to learn how emotions can be expressed and recognised via playing games in a virtual world. The platform includes analysis of users' gestures, facial, and vocal expressions using standard microphone and web-cam or a depth sensor, training through games, text communication with peers, animation, video and audio clips. We present the state of play in realising such a serious g…
Introducing CALMED: Multimodal Annotated Dataset for Emotion Detection in Children with Autism
University of Galway（戈尔韦大学）；LORIA CNRS/INRIA/Université de Lorraine（洛林大学国家科学研究中心/法国国家信息与自动化研究所洛林计算机科学及其应用实验室）；Université Sorbonne Paris Nord（巴黎索邦北大学）；ASK - All Special Kids（ASK-所有特殊儿童）
Robots have great potential to facilitate future therapies for children on the autism spectrum. However, existing robots lack the ability to automatically perceive and respond to human affect, which is necessary for establishing and maintaining engaging interactions. Moreover, their inference challenge is made harder by the fact that many individuals with autism have atypical and unusually diverse styles of expressing their affective-cognitive states. To tackle the heterogeneity in behavioral cues of children with autism, we use the latest advances in deep learning to formulate a personalized …
University of Wisconsin–Madison（威斯康星大学麦迪逊分校）；University of Illinois Chicago（伊利诺伊大学芝加哥分校）
Social robots are emerging as learning companions for children, and research shows that they facilitate the development of interest and learning even through brief interactions. However, little is known about how such technologies might support these goals in authentic environments over long-term periods of use and interaction. We designed a learning companion robot capable of supporting children reading popular-science books by expressing social and informational commentaries. We deployed the robot in homes of 14 families with children aged 10-12 for four weeks during the summer. Our analysis…
The Nexus of AR/VR, AI, UI/UX, and Robotics Technologies in Enhancing Learning and Social Interaction for Children with Autism Spectrum Disorders: A Systematic Review
IMT Atlantique（IMT大西洋学院）；Brest University Hospital（布雷斯特大学医院）
Previous studies have suggested that being imitated by an adult is an effective intervention with children with autism and developmental delay. The purpose of this study is to investigate if an imitation game with a robot can arise interest from children and constitute an effective tool to be used in clinical activities. In this paper, we describe the design of our nursery rhyme imitation game, its implementation based on RGB image pose recognition and the preliminary tests we performed.
Autism Spectrum Disorder (ASD) is the area where many researches enduring like Magnetic Resonance Imaging (MRI), called diffusion tensor imaging, Early Start Denver Model (ESDM) to provide an easier life for the people diagnosed. After years and years of combined funding sources from public and private funding, these researches show great promises in recent years. In this paper, we have tried to show a way how children with Down Syndrome Autism can learn through game therapy. These game therapies have shown an immense number of improvements among those children to learn alphabets along with de…
A multimodal large language model for evidence-based autism spectrum disorder screening
Zhejiang Normal University（浙江师范大学）；State Grid Corporation of China（国家电网有限公司）；Huzhou Normal University（湖州师范学院）；Zhejiang University of Technology（浙江工业大学）
Universidad Peruana Cayetano Heredia（秘鲁卡耶塔诺·埃雷迪亚大学）；Johns Hopkins University（约翰斯·霍普金斯大学）
Autism is a developmental disorder that affects social interaction and communication of children. The gold standard diagnostic tools are very difficult to use and time consuming. However, diagnostic could be deduced from child gaze preferences by looking a video with social and abstract scenes. In this work, we propose an algorithm based on convolutional neural networks to predict gaze direction for a fast and effective autism diagnosis. Early results show that our algorithm achieves real-time response and robust high accuracy for prediction of gaze direction.
Surprising Performances of Students with Autism in Classroom with NAO Robot
School of Computer and Information Technology（计算机与信息学院）；Northeast Petroleum University（东北石油大学）；School of Computer Science and Engineering（计算机科学与工程学院）；Changshu Institute of Technology（常州职业技术学院）；Changshu Special Education School（常州特殊教育学校）；School of Chinese Language and Literature（中文语言文学学院）；Nanjing Normal University（南京师范大学）
Syracuse University（雪城大学）；UnityPoint Health（UnityPoint健康）；Stevens Institute of Technology（史蒂文斯理工学院）；Binghamton University（宾汉姆顿大学）；Texas Tech University（得克萨斯理工大学）
Application of humanoid robots has been common in the field of healthcare and education. It has been recurrently used to improve social behavior and mollify distress level among children with autism, cancer and cerebral palsy. This article discusses the same from a human factors perspective. It shows how people of different age and gender have a different opinion towards the application and acceptance of humanoid robots. Additionally, this article highlights the influence of cerebral condition and social interaction on a user behavior and attitude towards humanoid robots. Our study performed a…
Engagement in Human-Machine Interaction is the process by which entities participating in the interaction establish, maintain, and end their perceived connection. It is essential to monitor the engagement state of patients in various AI-based interactive healthcare paradigms. This includes medical conditions that alter social behavior such as Autism Spectrum Disorder (ASD) or Attention-Deficit/Hyperactivity Disorder (ADHD). Engagement is a multi-faceted construct which is composed of behavioral, emotional, and mental components. Previous research has neglected this multi-faceted nature of enga…
Beyond Usability: A UX Case Study on Using "Withdrawal Design" to Challenge Engagement Metrics in Social Robotics
Tsinghua University（清华大学）；Nanjing University of Chinese Medicine（南京中医药大学）；Nanjing University（南京大学）；University of Washington（华盛顿大学）；North China University of Science and Technology Health Science Center（华北理工大学健康科学中心）
Fostering data visualization literacy (DVL) as part of childhood education could lead to a more data literate society. However, most work in DVL for children relies on a more formal educational context (i.e., a teacher-led approach) that limits children's engagement with data to classroom-based environments and, consequently, children's ability to ask questions about and explore data on topics they find personally meaningful. We explore how a curiosity-driven, child-led approach can provide more agency to children when they are authoring data visualizations. This paper explores how informal le…
The coronavirus disease (COVID-19) pandemic affected our lives deeply, just like everyone else, the children also suffered from the restrictions due to COVID-19 affecting their education and social interactions with others, being restricted from play areas and schools for a long time. Although social robots provide a promising solution to support children in their education, healthcare and social interaction with others, the precautions due to COVID-19 also introduced new constraints in the social robotics research. In this paper, we will discuss the benefits and challenges encountered in chil…
EMooly: Supporting Autistic Children in Collaborative Social-Emotional Learning with Caregiver Participation through Interactive AI-infused and AR Activities
University of Waterloo（滑铁卢大学）；Southern University of Science and Technology（南方科技大学）；Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；National Research Council（国家研究委员会）
We have developed a system for automatic facial expression recognition, which runs on Google Glass and delivers real-time social cues to the wearer. We evaluate the system as a behavioral aid for children with Autism Spectrum Disorder (ASD), who can greatly benefit from real-time non-invasive emotional cues and are more sensitive to sensory input than neurotypically developing children. In addition, we present a mobile application that enables users of the wearable aid to review their videos along with auto-curated emotional information on the video playback bar. This integrates our learning a…
