How Do Students Interact with an LLM-powered Virtual Teaching Assistant in Different Educational Settings?
Georgia Institute of Technology（佐治亚理工学院）
Pratyusha Maiti, Ashok K. Goel
备注：
Accepted in the Seventeenth International Conference on Educational Data Mining (EDM) Workshop: Leveraging LLMs for Next Generation Educational Technologies, July 2024
Jill Watson, a virtual teaching assistant powered by LLMs, answers student questions and engages them in extended conversations on courseware provided by the instructors. In this paper, we analyze student interactions with Jill across multiple courses and colleges, focusing on the types and complexity of student questions based on Bloom's Revised Taxonomy and tool usage patterns. We find that, by supporting a wide range of cognitive demands, Jill encourages students to engage in sophisticated, higher-order cognitive questions. However, the frequency of usage varies significantly across deployments, and the types of questions asked depend on course-specific contexts. These findings pave the way for future work on AI-driven educational tools tailored to individual learning styles and course structure, potentially enhancing both the teaching and learning experience in classrooms.
A Prompt-Engineering Approach to Develop Scalable, Flexible, and Real-Time Hybrid Micro-Level Personalization in a General Purpose AI Teaching Assistant
Introductory hands-on courses such as our smartphone-based coding course, SuaCode require a lot of support for students to accomplish learning goals. Online environments make it even more difficult to get assistance especially more recently because of COVID-19. Given the multilingual context of SuaCode students - learners across 42 African countries that are mostly Anglophone or Francophone - in this work, we developed a bilingual Artificial Intelligence (AI) Teaching Assistant (TA) - Kwame - that provides answers to students' coding questions from SuaCode courses in English and French. Kwame …
Beyond Helpfulness: A Teaching-over-Solving Diagnostic for Measuring Educational Impact in LLM Tutors
Washington University in St. Louis（圣路易斯华盛顿大学）；Department of Operations Research and Engineering Management, Southern Methodist University（南卫理公会大学运筹学与工程管理系）
RAM2C: A Liberal Arts Educational Chatbot based on Retrieval-augmented Multi-role Multi-expert Collaboration
Center for Brain Inspired Computing Research (CBICR), Tsinghua University（清华大学脑启发计算研究中心）；Optical Memory National Engineering Research Center, Tsinghua University（清华大学光存储国家工程研究中心）；Tsinghua University-China Electronics Technology HIK Group Co. Joint Research Center for Brain-inspired Computing（清华大学-中国电子 hik 集团联合脑启发计算研究中心）；IDG/McGovern Institute for Brain Research, Tsinghua University（清华大学 IDG/麦戈文脑科学研究院）；Tsinghua University（清华大学）
University teachers often feel the need to try innovative learning technologies such as Learning Games to motivate the new generation of students. However, the typically limited resources of universities coupled with the high cost of designing and developing Learning Games result in it rarely being feasible to meet this need. To address this challenging problem, we have designed a framework that allows teachers to create their own Learning Games with very little or no help from developers and graphic designers. This framework, tested and validated by several university teachers, is suited to a…
The focus of this article are issues related to the assessment of the teacher, working in small schools. The author analyzes the educational activities ungraded school teacher, as one of the factors increasing the quality of teaching in a rural school. We have tried to consider other resources to successfully solve the problem of increasing the quality of education in rural schools. Teacher in a rural school is a key figure in the training, development and education of children. From his profession, qualification, ability to work with a small but different age student population is entirely de…
This project report attempts to evaluate the educational properties of the Python computer language, in practice. This is done by examining computer language evolution history, related scientific background work, the existing educational research on computer languages and Python's experimental application in higher secondary education in Greece, during first half of year 2002. This Thesis Report was delivered in advance of a thesis defense for a Masters/Doctorandus (MSc/Drs) title with the Amstel Institute/Universiteit van Amsterdam, during the same year.
The purpose of this study is to discuss instructional design and technology (IDT) model strategies for developing learning and teaching environments, based on philosophical approaches to educational technology theory. The study begins with a discussion of IDT models to define the history of educational technology or instructional technology theories, based on instructional strategies and improvements. In the study, authors discuss the strategies and steps that a design team should follow when designing learning environments in industry, business and military scenarios, based on the philosophy …
In this paper we present an innovative intelligent web-based computer-aided instruction system for foreign language learning: CSIEC (Computer Simulator in Educational Communication). This system can not only grammatically understand the sentences in English given from the users via Internet, but also reasonably and individually speak with the users. At first the related works in this research field are analyzed. Then we introduce the system goals and the system framework, i.e., the natural language understanding mechanism (NLML, NLOMJ and NLDB) and the communicational response (CR). Finally we…
Video annotation and analysis is an important activity for teaching with and about audiovisual media artifacts because it helps students to learn how to identify textual and formal connections in media products. But school teachers lack adequate tools for video annotation and analysis in media education that are easy-to-use, integrate into established teaching organization, and support quick collaborative work. To address these challenges, we followed a design-based research approach and conducted qualitative interviews with teachers to develop TRAVIS GO, a web application for simple and colla…
Unveiling User Perceptions in the Generative AI Era: A Sentiment-Driven Evaluation of AI Educational Apps' Role in Digital Transformation of e-Teaching
Islamic Azad University（伊斯兰阿扎德大学）；University of Isfahan（伊斯法罕大学）
Agentic AI Ecosystems in Higher Education: A Perspective on AI Agents to Emerging Inclusive, Agentic Multi-Agent AI Framework for Learning, Teaching and Institutional Intelligence
College of Computing and Data Science, Nanyang Technological University, NTU, Singapore（南洋理工大学计算机与数据科学学院）；DeepMed Ptd Ltd, India（印度DeepMed公司）
Virtual Worlds (VWs) are an emerging technology used by a growing number of educational institutions around the world. It is an environment, a way of learning and an educational tool that allows different levels of online interaction. In the course "Programming I", of the career Informatics Engineering at Universidad de Valparaíso, we conducted a pilot experience with the VW of Second Life, in order to evaluate the potential of using VWs in the teaching practice.
The use of open educational resources (OER) is gaining momentum in higher education institutions. This study sought to establish academics' perceptions and knowledge of OER for teaching and learning in an open distance e-learning (ODeL) university. The study also sought to establish how perceptions are formed. The inductive approach followed the lens of commodification to answer the research questions. The commodification phase allowed for a better understanding of the academics' prior knowledge, informers, academics behaviour about OER, and how they perceived OER to be useful for teaching and…
Teaching assistants (TAs) are heavily used in computer science courses as a way to handle high enrollment and still being able to offer students individual tutoring and detailed assessments. TAs are themselves students who take on this additional role in parallel with their own studies at the same institution. Previous research has shown that being a TA can be challenging but has mainly been conducted on TAs from a single institution or within a single course. This paper offers a multi-institutional, multi-national perspective of challenges that TAs in computer science face. This has been done…
Since underlying hardware components form the basis of trust in virtually any computing system, security failures in hardware pose a devastating threat to our daily lives. Hardware reverse engineering is commonly employed by security engineers in order to identify security vulnerabilities, to detect IP violations, or to conduct very-large-scale integration (VLSI) failure analysis. Even though industry and the scientific community demand experts with expertise in hardware reverse engineering, there is a lack of educational offerings, and existing training is almost entirely unstructured and on …
The COVID-19 pandemic significantly disrupted the educational sector. Faced with this life-threatening pandemic, educators had to swiftly pivot to an alternate form of course delivery without severely impacting the quality of the educational experience. Following the transition to online learning, educators had to grapple with a host of challenges. With interrupted face-to-face delivery, limited access to state-of-the-art labs, barriers with educational technologies, challenges of academic integrity, and obstacles with remote teamwork and student participation, creative solutions were urgently…
Conversational agents are becoming increasingly popular for supporting and facilitating learning. Conventional pedagogical agents are designed to play the role of human teachers by giving instructions to the students. In this paper, we investigate the use of conversational agents to support the 'learning-by-teaching' paradigm where the agent receives instructions from students. In particular, we introduce Curiosity Notebook: an educational application that harvests conversational interventions to facilitate students' learning. Recognizing such interventions can not only help in engaging studen…
Africa has a high student-to-teacher ratio which limits students' access to teachers. Consequently, students struggle to get answers to their questions. In this work, we extended Kwame, our previous AI teaching assistant, adapted it for science education, and deployed it as a web app. Kwame for Science answers questions of students based on the Integrated Science subject of the West African Senior Secondary Certificate Examination (WASSCE). Kwame for Science is a Sentence-BERT-based question-answering web app that displays 3 paragraphs as answers along with a confidence score in response to sc…
The paper describes creation and development of the educational online communication platform for teaching and learning foreign languages. The system is based on the time bank principle, allowing users to teach others their native tongue along with taking foreign language lessons. The system is based on the WebRTC technology, allowing users to access synchronized teaching materials along with seeing and hearing each other. The platform is free for the users with implemented gamification mechanics to motivate them. It is based on the freemium model, where the main functions are provided free of…
Carnegie Mellon University（卡内基梅隆大学）；University of California, Berkeley（加州大学伯克利分校）；University of Michigan（密歇根大学）
Recently, there have been increasing calls for computer science curricula to complement existing technical training with topics related to Fairness, Accountability, Transparency, and Ethics. In this paper, we present Value Card, an educational toolkit to inform students and practitioners of the social impacts of different machine learning models via deliberation. This paper presents an early use of our approach in a college-level computer science course. Through an in-class activity, we report empirical data for the initial effectiveness of our approach. Our results suggest that the use of the…
In the past few years, the use of new technologies and digital, educational material has been increasing. Owing to the situation brought about in Spain by the SARS-CoV-2 (COVID-19) pandemic, schools have closed and families were asked to confine at home since March 2020. During this period there has been a need of resources to be made available to the families and teachers so that they would be able to continue their teaching practice. Consequently, this study analyzes the importance of online methodologies and usage tendency of an educational resource example: The Smile and Learn platform. Th…
