With most recent estimates giving an incidence rate of 1 in 68 children in the United States, the autism spectrum disorder (ASD) is a growing public health crisis. Many of these children struggle to make eye contact, recognize facial expressions, and engage in social interactions. Today the standard for treatment of the core autism-related deficits focuses on a form of behavior training known as Applied Behavioral Analysis. To address perceived deficits in expression recognition, ABA approaches routinely involve the use of prompts such as flash cards for repetitive emotion recognition training via memorization. These techniques must be administered by trained practitioners and often at clinical centers that are far outnumbered by and out of reach from the many children and families in need of attention. Waitlists for access are up to 18 months long, and this wait may lead to children regressing down a path of isolation that worsens their long-term prognosis. There is an urgent need to innovate new methods of care delivery that can appropriately empower caregivers of children at risk or with a diagnosis of autism, and that capitalize on mobile tools and wearable devices for use outside of clinical settings.
Imitation plays an important role in development, being one of the precursors of social cognition. Even though some children with autism imitate spontaneously and other children with autism can learn to imitate, the dynamics of imitation is affected in the large majority of cases. Existing studies from the literature suggest that robots can be used to teach children with autism basic interaction skills like imitation. Based on these findings, in this study, we investigate if children with autism show more social engagement when interacting with an imitative robot (Fig 1) compared to a human pa…
Neurofeedback games are an effective and playful approach to enhance certain social and attentional capabilities in children with autism, which are promising to become widely accessible along with the commercialization of mobile EEG modules. However, little industry-based experiences are shared, regarding how to better design neurofeedback games to fine-tune their playability and user experiences for autistic children. In this paper, we review the experiences we gained from industry practice, in which a series of mobile EEG neurofeedback games have been developed for preschool autistic childre…
In this exploratory study we assessed how attitudes of children with autism spectrum disorder (ASD) towards robots together with children's autism-related social impairments are linked to indicators of children's preference of an interaction with a robot over an interaction with a person. We found that children with ASD have overall positive attitudes towards robots and that they often prefer interacting with a robot than with a person. Several of children's attitudes were linked to children's longer gazes towards a robot compared to a person. Autism-related social impairments were linked to m…
Autism spectrum disorder (ASD) is a developmental disorder that influences the communication and social behavior of a person in a way that those in the spectrum have difficulty in perceiving other people's facial expressions, as well as presenting and communicating emotions and affect via their own faces and bodies. Some efforts have been made to predict and improve children with ASD's affect states in play therapy, a common method to improve children's social skills via play and games. However, many previous works only used pre-trained models on benchmark emotion datasets and failed to consid…
Technische Universität München（慕尼黑工业大学）；University of Cambridge（剑桥大学）
Individuals with Autism Spectrum Conditions (ASC) have marked difficulties using verbal and non-verbal communication for social interaction. The running ASC-Inclusion project aims to help children with ASC by allowing them to learn how emotions can be expressed and recognised via playing games in a virtual world. The platform includes analysis of users' gestures, facial, and vocal expressions using standard microphone and web-cam or a depth sensor, training through games, text communication with peers, animation, video and audio clips. We present the state of play in realising such a serious g…
A good teaching method is incomprehensible for an autistic child. The autism spectrum disorder is a very diverse phenomenon. It is said that no two autistic children are the same. So, something that works for one child may not be fit for another. The same case is true for their education. Different children need to be approached with different teaching methods. But it is quite hard to identify the appropriate teaching method. As the term itself explains, the autism spectrum disorder is like a spectrum. There are multiple factors to determine the type of autism of a child. A child might even be…
There is a lack of autonomy on traditional Robot-Assisted Therapy systems interacting with children with autism. To overcome this limitation a supervised autonomous robot controller is being built. In this paper we present a multilayer reactive system within such controller. The goal of this Reactive system is to allow the robot to appropriately react to the child's behavior creating the illusion of being alive.
This research addressed whether interactions with social robots could facilitate joint attention of the autism spectrum disorder (ASD). Two conditions of initiators, namely 'Human' vs. 'Robot' were measured with 15 children with ASD and 15 age-matched typically developing (TD) children. Apart from fixation and gaze transition, a new longest common subsequence (LCS) approach was proposed to analyze eye-movement traces. Results revealed that children with ASD showed deficits of joint attention. Compared to the human agent, robot facilitate less fixations towards the targets, but it attracted mor…
University of Tehran（德黑兰大学）；Intelligent Systems Research Institute, SKKU（成均馆大学智能系统研究所）；Allameh Tabatabái University（阿拉梅塔巴塔巴伊大学）；Iran University of Medical Sciences（伊朗医科大学）；Shahid Beheshti University（沙希德贝赫什提大学）
Robot Assisted Therapy is a new paradigm in many therapies such as the therapy of children with autism spectrum disorder. In this paper we present the use of a parrot-like robot as an assistive tool in turn taking therapy. The therapy is designed in the form of a card game between a child with autism and a therapist or the robot. The intervention was implemented in a single subject study format and the effect sizes for different turn taking variables are calculated. The results show that the child robot interaction had larger effect size than the child trainer effect size in most of the turn t…
Social visual behavior, as a type of non-verbal communication, plays a central role in studying social cognitive processes in interactive and complex settings of autism therapy interventions. However, for social visual behavior analytics in children with autism, it is challenging to collect gaze data manually and evaluate them because it costs a lot of time and effort for human coders. In this paper, we introduce a social visual behavior analytics approach by quantifying the mutual gaze performance of children receiving play-based autism interventions using an automated mutual gaze detection f…
Visual Programming Tools (VPTs) allow users to create interactive media projects such as games and animations using visual representations of programming concepts. Although VPTs have been shown to have huge potential for teaching children with cognitive impairments including those with Autism Spectrum Condition (ASC), research has shown that existing VPTs may not be accessible to them. Therefore, this study proposes a set of recommendations for the design of accessible VPTs for children with ASC. Recommendations were initially gathered and validated by interviewing experts (n=7). The interview…
National University of Singapore（新加坡国立大学）；Ministry of Education, Singapore（新加坡教育部）
This paper reviews the use of socially interactive robots to assist in the therapy of children with autism. The extent to which the robots were successful in helping the children in their social, emotional, and communication deficits was investigated. Child-robot interactions were scrutinized with respect to the different target behaviors that are to be elicited from a child during therapy. These behaviors were thoroughly examined with respect to a childs development needs. Most importantly, experimental data from the surveyed works were extracted and analyzed in terms of the target behaviors …
Affect-aware socially assistive robotics (SAR) tutors have great potential to augment and democratize professional therapeutic interventions for children with autism spectrum disorders (ASD) from different socioeconomic backgrounds. However, the majority of research on SAR for ASD has been on teaching cognitive and/or social skills, not on addressing users' emotional needs for real-world social situations. To bridge that gap, this work aims to develop personalized affect-aware SAR tutors to help alleviate social anxiety and foster social grit-the growth mindset for social skill development-in …
Socially assistive robotics (SAR) has great potential to provide accessible, affordable, and personalized therapeutic interventions for children with autism spectrum disorders (ASD). However, human-robot interaction (HRI) methods are still limited in their ability to autonomously recognize and respond to behavioral cues, especially in atypical users and everyday settings. This work applies supervised machine learning algorithms to model user engagement in the context of long-term, in-home SAR interventions for children with ASD. Specifically, we present two types of engagement models for each …
Introducing CALMED: Multimodal Annotated Dataset for Emotion Detection in Children with Autism
University of Galway（戈尔韦大学）；LORIA CNRS/INRIA/Université de Lorraine（洛林大学国家科学研究中心/法国国家信息与自动化研究所洛林计算机科学及其应用实验室）；Université Sorbonne Paris Nord（巴黎索邦北大学）；ASK - All Special Kids（ASK-所有特殊儿童）
A multimodal large language model for evidence-based autism spectrum disorder screening
Zhejiang Normal University（浙江师范大学）；State Grid Corporation of China（国家电网有限公司）；Huzhou Normal University（湖州师范学院）；Zhejiang University of Technology（浙江工业大学）
Robots have great potential to facilitate future therapies for children on the autism spectrum. However, existing robots lack the ability to automatically perceive and respond to human affect, which is necessary for establishing and maintaining engaging interactions. Moreover, their inference challenge is made harder by the fact that many individuals with autism have atypical and unusually diverse styles of expressing their affective-cognitive states. To tackle the heterogeneity in behavioral cues of children with autism, we use the latest advances in deep learning to formulate a personalized …
Children with Autism Spectrum Disorder (ASD) experience deficits in verbal and nonverbal communication skills including motor control, turn-taking, and emotion recognition. Innovative technology, such as socially assistive robots, has shown to be a viable method for Autism therapy. This paper presents a novel robot-based music-therapy platform for modeling and improving the social responses and behaviors of children with ASD. Our autonomous social interactive system consists of three modules. We adopted Short-time Fourier Transform and Levenshtein distance to fulfill the design requirements: a…
University of Southern California（南加州大学）；Massachusetts Institute of Technology（麻省理工学院）
Affect-aware socially assistive robotics (SAR) has shown great potential for augmenting interventions for children with autism spectrum disorders (ASD). However, current SAR cannot yet perceive the unique and diverse set of atypical cognitive-affective behaviors from children with ASD in an automatic and personalized fashion in long-term (multi-session) real-world interactions. To bridge this gap, this work designed and validated personalized models of arousal and valence for children with ASD using a multi-session in-home dataset of SAR interventions. By training machine learning (ML) algorit…
The Nexus of AR/VR, AI, UI/UX, and Robotics Technologies in Enhancing Learning and Social Interaction for Children with Autism Spectrum Disorders: A Systematic Review
Wearable devices hold great potential for promoting children's health and well-being. However, research on kids' wearables is sparse and often focuses on their use in the context of parental surveillance. To gain insight into the current landscape of kids' wearables, we surveyed 47 wearable devices marketed for children. We collected rich data on the functionality of these devices and assessed how different features satisfy parents' information needs, and identified opportunities for wearables to support children's needs and interests. We found that many kids' wearables are technologically sop…
Universidad Peruana Cayetano Heredia（秘鲁卡耶塔诺·埃雷迪亚大学）；Johns Hopkins University（约翰斯·霍普金斯大学）
Autism is a developmental disorder that affects social interaction and communication of children. The gold standard diagnostic tools are very difficult to use and time consuming. However, diagnostic could be deduced from child gaze preferences by looking a video with social and abstract scenes. In this work, we propose an algorithm based on convolutional neural networks to predict gaze direction for a fast and effective autism diagnosis. Early results show that our algorithm achieves real-time response and robust high accuracy for prediction of gaze direction.
In recent years, much focus has been put on employing technology to make novel behavioural aids for those with autism. Most of these are digital adaptations of tools used in standard behavioural therapy to enforce normative skills. These digital counterparts are often used outside of both the larger therapeutic context and the real world, in which the learned skills might apply. To address this, we are designing a system of automatic expression recognition on wearable devices that integrates directly into the families daily social interactions, to give children and their caregivers the tools a…
Autism Spectrum Disorder (ASD) is the area where many researches enduring like Magnetic Resonance Imaging (MRI), called diffusion tensor imaging, Early Start Denver Model (ESDM) to provide an easier life for the people diagnosed. After years and years of combined funding sources from public and private funding, these researches show great promises in recent years. In this paper, we have tried to show a way how children with Down Syndrome Autism can learn through game therapy. These game therapies have shown an immense number of improvements among those children to learn alphabets along with de…
We have developed a system for automatic facial expression recognition, which runs on Google Glass and delivers real-time social cues to the wearer. We evaluate the system as a behavioral aid for children with Autism Spectrum Disorder (ASD), who can greatly benefit from real-time non-invasive emotional cues and are more sensitive to sensory input than neurotypically developing children. In addition, we present a mobile application that enables users of the wearable aid to review their videos along with auto-curated emotional information on the video playback bar. This integrates our learning a…
Deep Learning Based Approach to Enhanced Recognition of Emotions and Behavioral Patterns of Autistic Children
Department of Information Technology（信息科技系）；Sri Lanka Institute of Information Technology（斯里兰卡信息科技研究所）；Department of Computer Systems Engineering（计算机系统工程系）
The coronavirus disease (COVID-19) pandemic affected our lives deeply, just like everyone else, the children also suffered from the restrictions due to COVID-19 affecting their education and social interactions with others, being restricted from play areas and schools for a long time. Although social robots provide a promising solution to support children in their education, healthcare and social interaction with others, the precautions due to COVID-19 also introduced new constraints in the social robotics research. In this paper, we will discuss the benefits and challenges encountered in chil…
EMooly: Supporting Autistic Children in Collaborative Social-Emotional Learning with Caregiver Participation through Interactive AI-infused and AR Activities
University of Waterloo（滑铁卢大学）；Southern University of Science and Technology（南方科技大学）；Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；National Research Council（国家研究委员会）
Surprising Performances of Students with Autism in Classroom with NAO Robot
School of Computer and Information Technology（计算机与信息学院）；Northeast Petroleum University（东北石油大学）；School of Computer Science and Engineering（计算机科学与工程学院）；Changshu Institute of Technology（常州职业技术学院）；Changshu Special Education School（常州特殊教育学校）；School of Chinese Language and Literature（中文语言文学学院）；Nanjing Normal University（南京师范大学）
Syracuse University（雪城大学）；UnityPoint Health（UnityPoint健康）；Stevens Institute of Technology（史蒂文斯理工学院）；Binghamton University（宾汉姆顿大学）；Texas Tech University（得克萨斯理工大学）
Application of humanoid robots has been common in the field of healthcare and education. It has been recurrently used to improve social behavior and mollify distress level among children with autism, cancer and cerebral palsy. This article discusses the same from a human factors perspective. It shows how people of different age and gender have a different opinion towards the application and acceptance of humanoid robots. Additionally, this article highlights the influence of cerebral condition and social interaction on a user behavior and attitude towards humanoid robots. Our study performed a…
Eye contact is a crucial element of non-verbal communication that signifies interest, attention, and participation in social interactions. As a result, measures of eye contact arise in a variety of applications such as the assessment of the social communication skills of children at risk for developmental disorders such as autism, or the analysis of turn-taking and social roles during group meetings. However, the automated measurement of visual attention during naturalistic social interactions is challenging due to the difficulty of estimating a subject's looking direction from video. This pap…
In current youth-care programs, children with needs (mental health, family issues, learning disabilities, and autism) receive support from youth and family experts as one-to-one assistance at schools or hospitals. Occasionally, social robots have featured in such settings as support roles in a one-to-one interaction with the child. In this paper, we suggest the development of a symbiotic framework for real-time Emotional Support (ES) with social robots Knowledge Graphs (KG). By augmenting a domain-specific corpus from the literature on ES for children (between the age of 8 and 12) and providin…
Beyond Usability: A UX Case Study on Using "Withdrawal Design" to Challenge Engagement Metrics in Social Robotics
Tsinghua University（清华大学）；Nanjing University of Chinese Medicine（南京中医药大学）；Nanjing University（南京大学）；University of Washington（华盛顿大学）；North China University of Science and Technology Health Science Center（华北理工大学健康科学中心）
