CATS黑客：创建和完善网络安全概念清单的测试项目
The CATS Hackathon: Creating and Refining Test Items for Cybersecurity Concept Inventories
摘要：
在2018年2月的两天里，来自政府和行业的17名网络安全教育工作者和专业人士举行了一次“黑客会议”，以完善现有的多项选择题测试项目草案，并为网络安全概念清单（CCI）和网络安全课程评估（CCA）创建新的项目，作为网络安全评估工具（CATS）项目的一部分。我们报告了CATS黑客马拉松的结果，讨论了我们用来开发测试项目的方法，强调了样本测试项目在这个过程中的演变，并为其他希望组织类似黑客马拉松的人提供了建议。每个测试项目包含一个场景，问题干，和五个答案选择。在黑客训练期间，参与者组成团队，（1）生成新的场景和问题词干，（2）将CCI项目扩展为CCA项目，并为新的场景和词干生成新的答案选择，以及（3）审查和改进CCA测试项目草案。CATS项目为评估和评价教育实践提供了严格的循证工具;这些工具可以帮助确定在网络安全教学中有效的方法和内容。CCI衡量学生在该领域的第一门课程后对网络安全基本概念的理解程度-特别是对抗性思维。CCA衡量学生在完成完整的网络安全课程后对核心概念的理解程度。
英文摘要：
For two days in February 2018, 17 cybersecurity educators and professionals from government and industry met in a "hackathon" to refine existing draft multiple-choice test items, and to create new ones, for a Cybersecurity Concept Inventory (CCI) and Cybersecurity Curriculum Assessment (CCA) being developed as part of the Cybersecurity Assessment Tools (CATS) Project. We report on the results of the CATS Hackathon, discussing the methods we used to develop test items, highlighting the evolution of a sample test item through this process, and offering suggestions to others who may wish to organize similar hackathons. Each test item embodies a scenario, question stem, and five answer choices. During the Hackathon, participants organized into teams to (1) Generate new scenarios and question stems, (2) Extend CCI items into CCA items, and generate new answer choices for new scenarios and stems, and (3) Review and refine draft CCA test items. The CATS Project provides rigorous evidence-based instruments for assessing and evaluating educational practices; these instruments can help identify pedagogies and content that are effective in teaching cybersecurity. The CCI measures how well students understand basic concepts in cybersecurity---especially adversarial thinking---after a first course in the field. The CCA measures how well students understand core concepts after completing a full cybersecurity curriculum.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
