基于注视的主动式AI辅助儿童图画探索

中文摘要 AI 辅助

基于大语言模型（LLM）的主动式辅助在人机交互（HCI）领域受到越来越多的关注。然而，以往关于主动式LLM辅助的研究大多聚焦于成年用户和任务导向场景，尚未探讨此类系统如何支持儿童——儿童的兴趣和需求往往通过注视等非语言行为而非明确请求来表达。本研究聚焦于儿童图画探索中主动辅助的两个关键挑战：何时提供辅助以及基于儿童的非语言行为提供何种辅助。为应对这些挑战，我们引入了Ollie，一个基于注视的主动式人工智能（AI）助手，它能根据儿童注视的位置提供简短的叙事描述。Ollie利用儿童的注视来估计其注意力，识别其当前的视觉焦点，并选择一个相关的图画区域供LLM进行口头描述。在受试者内实验中，我们将基于注视的辅助与随机辅助进行了比较。结果表明，基于注视的辅助能让儿童更长时间地保持对当前焦点的注意力，并更有效地引导他们关注相关的图画区域。儿童、家长以及一位参与的幼儿园教师对Ollie给予了积极评价，认为与随机辅助相比，它更符合儿童的兴趣。这项工作展示了将注视作为隐式输入用于儿童主动式AI辅助的可行性，并为未来以儿童为中心的AI系统提供了设计启示。

英文摘要

Proactive assistance with large language models (LLMs) has received growing attention in the human computer interaction (HCI) community. However, most past work on proactive LLMs' assistance has focused on adult users and task-oriented settings, leaving open how such systems could support children, whose interests and needs are often expressed through gaze and other nonverbal behaviors rather than explicit requests. In this study, we focus on two key challenges of proactive assistance in children's picture exploration: when to provide assistance and what assistance to provide based on children's nonverbal behaviors. To address these challenges, we introduce Ollie, a gaze-informed proactive artificial intelligence (AI) assistant that offers short narrative descriptions based on where a child is looking. Ollie uses children's gaze to estimate their attention, identify their current visual focus, and select a related picture region for the LLM to verbally describe. In a within-subject experiment, we compared gaze-informed assistance with random assistance. Results show that gaze-informed assistance kept children's attention on their current focus for a longer period of time, and guided them more effectively to related picture regions. Children, parents, and a participating kindergarten teacher viewed Ollie positively and consider that it better matched children's interests when compared with the random assistance. This work shows the feasibility of using gaze as an implicit input for proactive AI assistance for children and provides design implications for future child-centered AI systems.

发表机构

• Saarland University（萨尔兰大学）
• Tübingen Digital Teaching Lab (TüDiLab)（蒂宾根数字教学实验室）
• Leibniz-Institut für Wissensmedien (IWM Tübingen)（莱布尼茨知识媒体研究所）
• Research Institute for Information Technology, Kyushu University（九州大学信息技术研究院）

机构由 AI 辅助整理，请以论文原文为准。