arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2405.06203cs.AI
收藏

使用机器学习方法增强具身学习环境中互动分析的第一步

A First Step in Using Machine Learning Methods to Enhance Interaction Analysis for Embodied Learning Environments

Vanderbilt University(范德堡大学);Indiana University(印第安纳大学)

Joyce Fonteles, Eduardo Davalos, Ashwin T. S., Yike Zhang, Mengxi Zhou, Efrat Ayalon, Alicia Lane, Selena Steinberg, Gabriella Anton, Joshua Danish, Noel Enyedy, Gautam Biswas

AI总结:

本文提出结合机器学习与多模态学习分析支持 IA,通过时间轴可视化学生状态、动作、凝视、情感和移动,并在光合作用场景中验证其辅助识别关键学习时刻的可行性。

摘要:

研究儿童在混合现实环境中的具身学习(他们在其中协作模拟科学过程),需要分析复杂的多模态数据,以解释其学习与协调行为。学习科学家已经开发了用于分析此类数据的 Interaction Analysis(IA,互动分析)方法,但这要求研究人员观看数小时视频,以提取并解释学生的学习模式。本研究旨在简化研究人员的任务,使用机器学习和多模态学习分析来支持 IA 流程。本研究结合机器学习算法与多模态分析,以支持并简化研究人员的工作,使其通过学生在模拟场景中的动作、凝视和情感反应,全面理解学生的科学参与。为促进有效的研究人员—AI 协作,我们提出一项初步案例研究,以确定在时间轴上可视化呈现学生状态、行动、凝视、情感和移动的可行性。案例研究聚焦于学生学习光合作用的特定科学场景。该时间轴使我们能够考察由多模态分析和互动分析识别出的关键学习时刻之间的一致性,并揭示关于学生时间性学习进展的洞见。

英文摘要:

Investigating children's embodied learning in mixed-reality environments, where they collaboratively simulate scientific processes, requires analyzing complex multimodal data to interpret their learning and coordination behaviors. Learning scientists have developed Interaction Analysis (IA) methodologies for analyzing such data, but this requires researchers to watch hours of videos to extract and interpret students' learning patterns. Our study aims to simplify researchers' tasks, using Machine Learning and Multimodal Learning Analytics to support the IA processes. Our study combines machine learning algorithms and multimodal analyses to support and streamline researcher efforts in developing a comprehensive understanding of students' scientific engagement through their movements, gaze, and affective responses in a simulated scenario. To facilitate an effective researcher-AI partnership, we present an initial case study to determine the feasibility of visually representing students' states, actions, gaze, affect, and movement on a timeline. Our case study focuses on a specific science scenario where students learn about photosynthesis. The timeline allows us to investigate the alignment of critical learning moments identified by multimodal and interaction analysis, and uncover insights into students' temporal learning progressions.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑