Kwame for Science在西非科学教育中作为AI教学助手的真实世界部署与评估
Real-World Deployment and Evaluation of Kwame for Science, An AI Teaching Assistant for Science Education in West Africa
AI总结：
本研究将双语AI教学助手Kwame适配于科学教育并部署为Web应用，基于WASSCE科目提供知识段落与历年真题，8个月覆盖32国750用户，前3准确率87.2%，为非洲可扩展远程教育提供新方案。
摘要：
非洲的师生比例较高，这限制了学生从教师那里获得学习支持，例如教育问答。在本研究中，我们扩展了Kwame——一个用于编程教育的双语AI教学助手，将其适配于科学教育，并作为Web应用程序部署。Kwame for Science基于西非高中毕业证书考试（WASSCE）的Integrated Science科目，提供来自精心整理的知识源的段落以及相关的历年国家考试题，作为学生问题的答案。此外，学生可以查看历年国家考试题及其答案，并按年份、题型和主题进行筛选，这些主题由我们开发的主题检测模型自动分类（91%的未加权平均召回率）。我们在真实世界中部署了Kwame for Science，历时8个月，覆盖32个国家（其中15个在非洲）的750名用户，共提出1.5K个问题。我们的评估显示，前3准确率为87.2%（n=109个问题），这意味着Kwame for Science有很高的概率在显示的3个答案中提供至少一个有用的答案。我们对模型错误回答问题的原因进行了分类，为未来改进提供见解。我们还分享了此类工具在开发、部署和人机交互组件方面的挑战和经验，以帮助其他研究人员部署类似工具。作为非洲背景下首创的工具，Kwame for Science有望为非洲数百万人提供可扩展、经济高效且高质量的远程教育。
英文摘要：
Africa has a high student-to-teacher ratio which limits students' access to teachers for learning support such as educational question answering. In this work, we extended Kwame, a bilingual AI teaching assistant for coding education, adapted it for science education, and deployed it as a web app. Kwame for Science provides passages from well-curated knowledge sources and related past national exam questions as answers to questions from students based on the Integrated Science subject of the West African Senior Secondary Certificate Examination (WASSCE). Furthermore, students can view past national exam questions along with their answers and filter by year, question type, and topics that were automatically categorized by a topic detection model which we developed (91% unweighted average recall). We deployed Kwame for Science in the real world over 8 months and had 750 users across 32 countries (15 in Africa) and 1.5K questions asked. Our evaluation showed an 87.2% top 3 accuracy (n=109 questions) implying that Kwame for Science has a high chance of giving at least one useful answer among the 3 displayed. We categorized the reasons the model incorrectly answered questions to provide insights for future improvements. We also share challenges and lessons with the development, deployment, and human-computer interaction component of such a tool to enable other researchers to deploy similar tools. With a first-of-its-kind tool within the African context, Kwame for Science has the potential to enable the delivery of scalable, cost-effective, and quality remote education to millions of people across Africa.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
