AIriskEval-edu: 用于AI辅助K-12教育解释风险评估的新数据集
AIriskEval-edu: New Dataset for Risk Assessment in AI-mediated K-12 Educational Explanations
AI总结：
提出AIriskEval-edu-db2数据集，包含1639条解释及结构化可解释性标注，用于训练基于LLM的审计员评估K-12教学内容中的教学风险，涵盖五个维度。
摘要：
本文介绍了AIriskEval-edu-db2，这是一个新数据集，旨在训练和评估基于LLM的审计员，用于K-12年级教学内容中可解释的教学风险评估。该数据集包含来自170个精心策划的ScienceQA问题的1639条解释，涵盖科学、语言艺术和社会科学。对于每个问题，数据集包含由人类教师编写的解释以及由LLM模拟的教师档案生成的11条解释，这些档案与不同的教学风险相关。我们提出了一个与既定教育标准一致的综合风险评分标准，涵盖五个互补维度：事实准确性、深度和完整性、重点和相关性、学生水平适当性以及意识形态偏见。一个关键贡献是添加了785条带有结构化可解释性注释的解释，包括风险定位和风险描述。这些注释通过半自动过程生成，并经过专家教师验证。最后，我们进行了验证实验，比较了最先进的专有模型与轻量级本地Llama 3.1 8B模型在教学风险检测和可解释性评估方面的表现。这些实验评估了在AIriskEval-edu-db2上进行监督微调是否能使本地可部署模型接近或超越更强的前沿模型，同时保护教育审计和评估任务中的隐私。
英文摘要：
This work introduces AIriskEval-edu-db2, a new dataset designed to train and evaluate auditors based on LLMs for an explainable pedagogical risk assessment in instructional content for grades K-12. The dataset comprises 1,639 explanations from 170 curated ScienceQA questions, covering science, language arts, and social sciences. For each question, the dataset includes an explanation written by a human teacher alongside 11 explanations generated by LLM-simulated teacher profiles associated with distinct pedagogical risks. We propose a comprehensive risk rubric aligned with established educational standards that covers five complementary dimensions: factual precision, depth and completeness, focus and relevance, student-level appropriateness, and ideological bias. A key contribution is the addition of 785 explanations with structured explainability annotations, including risk localization and risk description. The annotations are produced through a semi-automatic process with expert teacher validation. Finally, we present validation experiments comparing state-of-the-art proprietary models with a lightweight local Llama 3.1 8B model in both the pedagogical risk detection and the explainability assessment. These experiments evaluate whether supervised fine-tuning on AIriskEval-edu-db2 enables a locally deployable model to approach or outperform stronger frontier models while preserving privacy in educational auditing and assessment tasks.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
