VerAs:先验证后评估STEM实验室报告
VerAs: Verify then Assess STEM Lab Reports
AI总结:
针对STEM长篇实验室报告自动评估的挑战,提出端到端神经架构VerAs,通过分离验证与评估模块,有效提升评分准确性和一致性。
摘要:
随着STEM教育对批判性思维技能的关注日益增加,科学写作在强调探究技能的课程中扮演着越来越重要的角色。最近发布的一个数据集包含两组来自探究式物理课程的大学水平实验室报告,该数据集依赖于利用多个维度的分析性评估量表,这些维度指定了学科知识和优秀解释的一般组成部分。每个分析维度都按6分制进行评估,以向学生提供详细的反馈,帮助他们提高科学写作技能。人工评估可能很慢,并且在大班级中难以校准以确保所有学生的一致性。尽管在STEM学科的开放式问题自动评估方面存在大量工作,但对于像实验室报告这样的长篇写作的研究却少得多。我们提出了一种端到端的神经架构,该架构具有独立的验证器和评估模块,灵感来自开放领域问答(OpenQA)的方法。VerAs首先验证报告是否包含与给定量表维度相关的任何内容,如果包含,则评估相关句子。在实验室报告上,VerAs优于基于OpenQA系统或自动作文评分(AES)的多个基线。VerAs在中学物理作文的分析性量表上也表现良好。
英文摘要:
With an increasing focus in STEM education on critical thinking skills, science writing plays an ever more important role in curricula that stress inquiry skills. A recently published dataset of two sets of college level lab reports from an inquiry-based physics curriculum relies on analytic assessment rubrics that utilize multiple dimensions, specifying subject matter knowledge and general components of good explanations. Each analytic dimension is assessed on a 6-point scale, to provide detailed feedback to students that can help them improve their science writing skills. Manual assessment can be slow, and difficult to calibrate for consistency across all students in large classes. While much work exists on automated assessment of open-ended questions in STEM subjects, there has been far less work on long-form writing such as lab reports. We present an end-to-end neural architecture that has separate verifier and assessment modules, inspired by approaches to Open Domain Question Answering (OpenQA). VerAs first verifies whether a report contains any content relevant to a given rubric dimension, and if so, assesses the relevant sentences. On the lab reports, VerAs outperforms multiple baselines based on OpenQA systems or Automated Essay Scoring (AES). VerAs also performs well on an analytic rubric for middle school physics essays.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。