Purpose: When a learner fails to reach a milestone, educators often wonder if there had been any warning signs that could have allowed them to intervene sooner. Machine learning can predict which students are at risk of failing a high-stakes certification exam. If predictions can be made well in advance of the exam, then educators can meaningfully intervene before students take the exam to reduce the chances of a failing score.
Methods: Using already-collected, first-year student assessment data from five cohorts in a Master of Physician Assistant Studies program, the authors implement an "adaptive minimum match" version of the k-nearest neighbors algorithm (AMMKNN), using changing numbers of neighbors to predict each student's future exam scores on the Physician Assistant National Certifying Examination (PANCE). Validation occurred in two ways: Leave-one-out cross-validation (LOOCV) and evaluating the predictions in a new cohort.
Results: AMMKNN achieved an accuracy of 93% in LOOCV. AMMKNN generates a predicted PANCE score for each student, one year before they are scheduled to take the exam. Students can then be classified into extra support, optional extra support, or no extra support groups. The educator then has one year to provide the appropriate customized support to each category of student.
Conclusions: Predictive analytics can identify at-risk students, so they can receive additional support or remediation when preparing for high-stakes certification exams. Educators can use the included methods and code to generate predicted test outcomes for students. The authors recommend that educators use this or similar predictive methods responsibly and transparently, as one of many tools used to support students.
Are LLMs Ready for Computer Science Education? A Cross-Domain, Cross-Lingual and Cognitive-Level Evaluation Using Professional Certification Exams
Zhongshan Ophthalmic Center, Sun Yat-sen University（中山大学中山眼科中心）；Faculty of Data Science, City University of Macau（澳门城市大学数据科学学院）；Department of Accounting, Faculty of Business and Economics, The University of Melbourne（墨尔本大学商学院会计系）；School of Information Technology, Faculty of Business and Hospitality, Torrens University Australia（托伦斯大学商业与酒店管理学院信息技术学院）
Performance of ChatGPT on the US Fundamentals of Engineering Exam: Comprehensive Assessment of Proficiency and Potential Implications for Professional Environmental Engineering Practice
IIHR – Hydroscience & Engineering, University of Iowa（爱荷华大学IIHR水文科学与工程研究所）；Civil and Environmental Engineering, University of Iowa（爱荷华大学土木与环境工程学院）；Electrical and Computer Engineering, University of Iowa（爱荷华大学电气与计算机工程学院）
本文评估基于 GPT-4 的 ChatGPT 在美国 FE 环境考试中的表现，发现非侵入式提示词修改可显著提升准确率，并探讨其教育应用、局限及考试诚信意义。
Prompt Engineering for Responsible Generative AI Use in African Education: A Report from a Three-Day Training Series
Mampong Technical College of Education（曼庞教育技术学院）；Northern Cape Provincial Department of Education（北开普省教育部）；Bagabaga College of Education（巴加巴加教育学院）；Gambaga College of Education（甘巴加教育学院）；St. Joseph’s College of Education（圣约瑟夫教育学院）；Federal University of Transportation, Daura（道拉联邦交通大学）；University of Education, Winneba（温尼巴教育大学）；University of Manitoba（曼尼托巴大学）；University for Development Studies（发展研究大学）；JET Education Services（JET教育服务机构）；Generative AI for Education and Research in Africa (GenAI-ERA)（非洲生成式人工智能教育与研究组织（GenAI-ERA））；National University of Lesotho（莱索托国立大学）；University of Georgia（佐治亚大学）
