A good teaching method is incomprehensible for an autistic child. The autism spectrum disorder is a very diverse phenomenon. It is said that no two autistic children are the same. So, something that works for one child may not be fit for another. The same case is true for their education. Different children need to be approached with different teaching methods. But it is quite hard to identify the appropriate teaching method. As the term itself explains, the autism spectrum disorder is like a spectrum. There are multiple factors to determine the type of autism of a child. A child might even be diagnosed with autism at the age of 9. Such a varied group of children of different ages, but specialized educational institutions still tend to them more or less the same way. This is where machine learning techniques can be applied to find a better way to identify a suitable teaching method for each of them. By analyzing their physical, verbal and behavioral performance, the proper teaching method can be suggested much more precisely compared to a diagnosis result. As a result, more children with autistic spectrum disorder can get better education that suits their needs the best.
Neurofeedback games are an effective and playful approach to enhance certain social and attentional capabilities in children with autism, which are promising to become widely accessible along with the commercialization of mobile EEG modules. However, little industry-based experiences are shared, regarding how to better design neurofeedback games to fine-tune their playability and user experiences for autistic children. In this paper, we review the experiences we gained from industry practice, in which a series of mobile EEG neurofeedback games have been developed for preschool autistic childre…
Social visual behavior, as a type of non-verbal communication, plays a central role in studying social cognitive processes in interactive and complex settings of autism therapy interventions. However, for social visual behavior analytics in children with autism, it is challenging to collect gaze data manually and evaluate them because it costs a lot of time and effort for human coders. In this paper, we introduce a social visual behavior analytics approach by quantifying the mutual gaze performance of children receiving play-based autism interventions using an automated mutual gaze detection f…
Imitation plays an important role in development, being one of the precursors of social cognition. Even though some children with autism imitate spontaneously and other children with autism can learn to imitate, the dynamics of imitation is affected in the large majority of cases. Existing studies from the literature suggest that robots can be used to teach children with autism basic interaction skills like imitation. Based on these findings, in this study, we investigate if children with autism show more social engagement when interacting with an imitative robot (Fig 1) compared to a human pa…
Visual Programming Tools (VPTs) allow users to create interactive media projects such as games and animations using visual representations of programming concepts. Although VPTs have been shown to have huge potential for teaching children with cognitive impairments including those with Autism Spectrum Condition (ASC), research has shown that existing VPTs may not be accessible to them. Therefore, this study proposes a set of recommendations for the design of accessible VPTs for children with ASC. Recommendations were initially gathered and validated by interviewing experts (n=7). The interview…
In this exploratory study we assessed how attitudes of children with autism spectrum disorder (ASD) towards robots together with children's autism-related social impairments are linked to indicators of children's preference of an interaction with a robot over an interaction with a person. We found that children with ASD have overall positive attitudes towards robots and that they often prefer interacting with a robot than with a person. Several of children's attitudes were linked to children's longer gazes towards a robot compared to a person. Autism-related social impairments were linked to m…
Children with Autism Spectrum Disorder (ASD) experience deficits in verbal and nonverbal communication skills including motor control, turn-taking, and emotion recognition. Innovative technology, such as socially assistive robots, has shown to be a viable method for Autism therapy. This paper presents a novel robot-based music-therapy platform for modeling and improving the social responses and behaviors of children with ASD. Our autonomous social interactive system consists of three modules. We adopted Short-time Fourier Transform and Levenshtein distance to fulfill the design requirements: a…
This research addressed whether interactions with social robots could facilitate joint attention of the autism spectrum disorder (ASD). Two conditions of initiators, namely 'Human' vs. 'Robot' were measured with 15 children with ASD and 15 age-matched typically developing (TD) children. Apart from fixation and gaze transition, a new longest common subsequence (LCS) approach was proposed to analyze eye-movement traces. Results revealed that children with ASD showed deficits of joint attention. Compared to the human agent, robot facilitate less fixations towards the targets, but it attracted mor…
University of Tehran（德黑兰大学）；Intelligent Systems Research Institute, SKKU（成均馆大学智能系统研究所）；Allameh Tabatabái University（阿拉梅塔巴塔巴伊大学）；Iran University of Medical Sciences（伊朗医科大学）；Shahid Beheshti University（沙希德贝赫什提大学）
Robot Assisted Therapy is a new paradigm in many therapies such as the therapy of children with autism spectrum disorder. In this paper we present the use of a parrot-like robot as an assistive tool in turn taking therapy. The therapy is designed in the form of a card game between a child with autism and a therapist or the robot. The intervention was implemented in a single subject study format and the effect sizes for different turn taking variables are calculated. The results show that the child robot interaction had larger effect size than the child trainer effect size in most of the turn t…
With most recent estimates giving an incidence rate of 1 in 68 children in the United States, the autism spectrum disorder (ASD) is a growing public health crisis. Many of these children struggle to make eye contact, recognize facial expressions, and engage in social interactions. Today the standard for treatment of the core autism-related deficits focuses on a form of behavior training known as Applied Behavioral Analysis. To address perceived deficits in expression recognition, ABA approaches routinely involve the use of prompts such as flash cards for repetitive emotion recognition training…
Affect-aware socially assistive robotics (SAR) tutors have great potential to augment and democratize professional therapeutic interventions for children with autism spectrum disorders (ASD) from different socioeconomic backgrounds. However, the majority of research on SAR for ASD has been on teaching cognitive and/or social skills, not on addressing users' emotional needs for real-world social situations. To bridge that gap, this work aims to develop personalized affect-aware SAR tutors to help alleviate social anxiety and foster social grit-the growth mindset for social skill development-in …
Autism spectrum disorder (ASD) is a developmental disorder that influences the communication and social behavior of a person in a way that those in the spectrum have difficulty in perceiving other people's facial expressions, as well as presenting and communicating emotions and affect via their own faces and bodies. Some efforts have been made to predict and improve children with ASD's affect states in play therapy, a common method to improve children's social skills via play and games. However, many previous works only used pre-trained models on benchmark emotion datasets and failed to consid…
Introducing CALMED: Multimodal Annotated Dataset for Emotion Detection in Children with Autism
University of Galway（戈尔韦大学）；LORIA CNRS/INRIA/Université de Lorraine（洛林大学国家科学研究中心/法国国家信息与自动化研究所洛林计算机科学及其应用实验室）；Université Sorbonne Paris Nord（巴黎索邦北大学）；ASK - All Special Kids（ASK-所有特殊儿童）
University of Southern California（南加州大学）；Massachusetts Institute of Technology（麻省理工学院）
Affect-aware socially assistive robotics (SAR) has shown great potential for augmenting interventions for children with autism spectrum disorders (ASD). However, current SAR cannot yet perceive the unique and diverse set of atypical cognitive-affective behaviors from children with ASD in an automatic and personalized fashion in long-term (multi-session) real-world interactions. To bridge this gap, this work designed and validated personalized models of arousal and valence for children with ASD using a multi-session in-home dataset of SAR interventions. By training machine learning (ML) algorit…
Socially assistive robotics (SAR) has great potential to provide accessible, affordable, and personalized therapeutic interventions for children with autism spectrum disorders (ASD). However, human-robot interaction (HRI) methods are still limited in their ability to autonomously recognize and respond to behavioral cues, especially in atypical users and everyday settings. This work applies supervised machine learning algorithms to model user engagement in the context of long-term, in-home SAR interventions for children with ASD. Specifically, we present two types of engagement models for each …
Robots have great potential to facilitate future therapies for children on the autism spectrum. However, existing robots lack the ability to automatically perceive and respond to human affect, which is necessary for establishing and maintaining engaging interactions. Moreover, their inference challenge is made harder by the fact that many individuals with autism have atypical and unusually diverse styles of expressing their affective-cognitive states. To tackle the heterogeneity in behavioral cues of children with autism, we use the latest advances in deep learning to formulate a personalized …
The Nexus of AR/VR, AI, UI/UX, and Robotics Technologies in Enhancing Learning and Social Interaction for Children with Autism Spectrum Disorders: A Systematic Review
Autism Spectrum Disorder (ASD) is the area where many researches enduring like Magnetic Resonance Imaging (MRI), called diffusion tensor imaging, Early Start Denver Model (ESDM) to provide an easier life for the people diagnosed. After years and years of combined funding sources from public and private funding, these researches show great promises in recent years. In this paper, we have tried to show a way how children with Down Syndrome Autism can learn through game therapy. These game therapies have shown an immense number of improvements among those children to learn alphabets along with de…
Research has shown that human-agent relationships form in similar ways to human-human relationships. Since children do not have the same critical analysis skills as adults (and may over-trust technology, for example), this relationship-formation is concerning. Nonetheless, little research investigates children's perceptions of conversational agents in-depth, and even less investigates how education might change these perceptions. We present K-12 workshops with associated conversational AI concepts to encourage healthier understanding and relationships with agents. Through studies with the curr…
College of Information Studies, University of Maryland（马里兰大学信息学院）
Iteratively building and testing machine learning models can help children develop creativity, flexibility, and comfort with machine learning and artificial intelligence. We explore how children use machine teaching interfaces with a team of 14 children (aged 7-13 years) and adult co-designers. Children trained image classifiers and tested each other's models for robustness. Our study illuminates how children reason about ML concepts, offering these insights for designing machine teaching experiences for children: (i) ML metrics (e.g. confidence scores) should be visible for experimentation; (…
Development of Immersive Virtual and Augmented Reality-Based Joint Attention Training Platform for Children with Autism
Indian Institute of Technology Delhi（印度德里理工学院）；Indian Institute of Technology Jodhpur（印度焦浦尔理工学院）；All India Institute of Medical Sciences Delhi（印度医学科学院德里分院）
University of Hawaiʻi at Mānoa（夏威夷大学马诺阿分校）；Stanford University School of Medicine（斯坦福大学医学院）
Autism Spectrum Disorder (autism) is a neurodevelopmental delay which affects at least 1 in 44 children. Like many neurological disorder phenotypes, the diagnostic features are observable, can be tracked over time, and can be managed or even eliminated through proper therapy and treatments. Yet, there are major bottlenecks in the diagnostic, therapeutic, and longitudinal tracking pipelines for autism and related delays, creating an opportunity for novel data science solutions to augment and transform existing workflows and provide access to services for more affected families. Several prior ef…
A multimodal large language model for evidence-based autism spectrum disorder screening
Zhejiang Normal University（浙江师范大学）；State Grid Corporation of China（国家电网有限公司）；Huzhou Normal University（湖州师范学院）；Zhejiang University of Technology（浙江工业大学）
Indian Institute of Technology Bombay（印度理工学院孟买分校）；University of Reading（雷丁大学）；De Montfort University（德蒙福特大学）；Sangath（桑加斯）
Screening for any of the Autism Spectrum Disorders is a complicated process often involving a hybrid of behavioural observations and questionnaire based tests. Typically carried out in a controlled setting, this process requires trained clinicians or psychiatrists for such assessments. Riding on the wave of technical advancement in mobile platforms, several attempts have been made at incorporating such assessments on mobile and tablet devices. In this paper we analyse videos generated using one such screening test. This paper reports the first use of the efficacy of using the observer's distan…
The coronavirus disease (COVID-19) pandemic affected our lives deeply, just like everyone else, the children also suffered from the restrictions due to COVID-19 affecting their education and social interactions with others, being restricted from play areas and schools for a long time. Although social robots provide a promising solution to support children in their education, healthcare and social interaction with others, the precautions due to COVID-19 also introduced new constraints in the social robotics research. In this paper, we will discuss the benefits and challenges encountered in chil…
EMooly: Supporting Autistic Children in Collaborative Social-Emotional Learning with Caregiver Participation through Interactive AI-infused and AR Activities
University of Waterloo（滑铁卢大学）；Southern University of Science and Technology（南方科技大学）；Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；National Research Council（国家研究委员会）
While the number of children with diagnosed autism spectrum disorder (ASD) continues to rise from year to year, there is still no universal approach to autism diagnosis and treatment. A great variety of different tools and approaches for the on-site diagnostic are available right now, however, a big percent of parents have no access to them and they tend to search for the available tools and correction programs on the Internet. Lack of money, absence of qualified specialists, and low level of trust to the correction methods are the main issues that affect the in-time diagnoses of ASD and which…
Exploring Gaze Pattern Differences Between Autistic and Neurotypical Children: Clustering, Visualisation, and Prediction
Singapore University of Technology and Design（新加坡科技设计大学）；A*STAR Institute for Infocomm Research（新加坡资讯通信研究院）；University of Wisconsin Madison（威斯康星大学麦迪逊分校）
George Washington University（乔治华盛顿大学）；Michigan Technological University（密歇根理工大学）；Georgia Institute of Technology（佐治亚理工学院）
The diagnosis of Autism Spectrum Disorder (ASD) in children is commonly accompanied by a diagnosis of sensory processing disorders as well. Abnormalities are usually reported in multiple sensory processing domains, showing a higher prevalence of unusual responses, particularly to tactile, auditory and visual stimuli. This paper discusses a novel robot-based framework designed to target sensory difficulties faced by children with ASD in a controlled setting. The setup consists of a number of sensory stations, together with robotic agents that navigate the stations and interact with the stimuli …
Deep Learning Based Approach to Enhanced Recognition of Emotions and Behavioral Patterns of Autistic Children
Department of Information Technology（信息科技系）；Sri Lanka Institute of Information Technology（斯里兰卡信息科技研究所）；Department of Computer Systems Engineering（计算机系统工程系）
In current youth-care programs, children with needs (mental health, family issues, learning disabilities, and autism) receive support from youth and family experts as one-to-one assistance at schools or hospitals. Occasionally, social robots have featured in such settings as support roles in a one-to-one interaction with the child. In this paper, we suggest the development of a symbiotic framework for real-time Emotional Support (ES) with social robots Knowledge Graphs (KG). By augmenting a domain-specific corpus from the literature on ES for children (between the age of 8 and 12) and providin…
