While multimodal large language models (MLLMs) are increasingly applied in human-centred AI systems, their ability to understand complex social interactions remains uncertain. We present an exploratory study on aligning MLLMs with speech-language pathologists (SLPs) in analysing joint attention in parent-child interactions, a key construct in early social-communicative development. Drawing on interviews and video annotations with three SLPs, we characterise how observational cues of gaze, action, and vocalisation inform their reasoning processes. We then test whether an MLLM can approximate this workflow through a two-stage prompting approach, separating observation from judgement. Our findings reveal that alignment is more robust at the observation layer, where experts share common descriptors, than at the judgement layer, where interpretive criteria diverge. We position this work as a case-based probe into expert-AI alignment in complex social behaviour, highlighting both the feasibility and the challenges of applying MLLMs to socially situated interaction analysis.
Navigating Marginalization: Toward Justice-Oriented Socio-Technical Design for Parent-Child Learning among Southeast Asian Immigrant Mothers in Taiwan
National Cheng Kung University(国立成功大学);University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);National Yang Ming Chiao Tung University(国立阳明交通大学);National University of Singapore(新加坡国立大学)
StoryBuddy: A Human-AI Collaborative Chatbot for Parent-Child Interactive Storytelling with Flexible Parental Involvement
University of Notre Dame(圣母大学);University of California, Irvine(加州大学欧文分校);Georgia Institute of Technology(佐治亚理工学院);Rensselaer Polytechnic Institute(伦斯勒理工学院);University of Washington(华盛顿大学);IBM Research(IBM研究院)
Towards Emotion Co-regulation with LLM-powered Socially Assistive Robots: Integrating LLM Prompts and Robotic Behaviors to Support Parent-Neurodivergent Child Dyads
Department of Industrial Design, Eindhoven University of Technology(工业设计系,埃因霍温理工大学);Department of Biomedical Engineering, Eindhoven University of Technology(生物医学工程系,埃因霍温理工大学);Department of Engineering, Institute of Science Tokyo(工程系,东京科学研究所);Department of Engineering, Utrecht University(工程系,乌得勒支大学)
LLM-TA: An LLM-Enhanced Thematic Analysis Pipeline for Transcripts from Parents of Children with Congenital Heart Disease
School of Information, UT Austin(德克萨斯大学奥斯汀分校信息学院);College of Natural Sciences, UT Austin(德克萨斯大学奥斯汀分校自然科学学院);Graphen, Inc.(格拉芬公司);Vanderbilt University Medical Center(范德堡大学医学中心);Pediatric Heart Institute at Monroe Carell Jr. Children’s Hospital at Vanderbilt(范德堡大学门罗·卡雷尔 Jr.儿童医院儿科心脏研究所);Texas Center for Pediatric and Congenital Heart Disease(德克萨斯儿科与先天性心脏病中心);Dell Medical School, UT Austin(德克萨斯大学奥斯汀分校戴尔医学院)