This paper presents the first item response theory (IRT) analysis of the national data set on introductory, general education, college-level astronomy teaching using the Light and Spectroscopy Concept Inventory (LSCI). We used the difference between students' pre- and post-instruction IRT-estimated abilities as a measure of learning gain. This analysis provides deeper insights than prior publications into both the LSCI as an instrument and into the effectiveness of teaching and learning in introductory astronomy courses. Our IRT analysis supports the classical test theory findings of prior studies using the LSCI with this population. In particular, we found that students in classes that used active learning strategies at least 25\% of the time had average IRT-estimated learning gains that were approximately 1 logit larger than students in classes that spent less time on active learning strategies. We also found that instructors who want their classes to achieve an improvement in abilities of average $Δθ= 1$ logit must spend at least 25\% of class time on active learning strategies. However, our analysis also powerfully illustrates the lack of insight into student learning that is revealed by looking at a single measure of learning gain, such as average $Δθ$. Educators and researchers should also examine the distributions of students' abilities pre- and post-instruction in order to understand how many students actually achieved an improvement in their abilities and whether or not a majority of students have moved to post-abilities significantly greater than the national average.
The Force Concept Inventory Across Continents: Testing Q-Matrix Transferability and Cross-Cultural Differences in Mechanics Reasoning
Australian Catholic University(澳大利亚天主教大学);Technolyceum (Ministry of Education School)(技术学院(教育部学校));University of Johannesburg(约翰内斯堡大学);City University Macau(澳门城市大学);De La Salle University(德拉萨大学)
2009年,巴西国家中等教育评估(ENEM,Exame Nacional do Ensino Médio)发生了变化:从高中结束时的能力自我评估改为允许获得大学和学生资助的评估。从单一的普通考试,现在有四个方面的考试:数学,语言,自然科学和社会科学。建立了一个新的参考矩阵,其组成部分包括认知领域、能力、技能和知识对象;方法框架也发生了变化,现在使用项目反应理论来提供分数,并允许对不同年份的结果进行纵向比较,为监测巴西高中质量提供了条件。本文对2009年、2010年和2011年全国环境管理自然科学测试中讨论的问题进行了研究。定性变量被提出来表征项目,并对学生在物理项目中的反应数据进行了分析。定性分析揭示了这几年高考的特点:题目长,只有少数题目要求较复杂的推理,一个特点是解题技巧。对学生成绩的分析还表明,学习物理并没有达到,项目的正确答案百分比几乎总是很小,需要某种学科知识或需要使用数学推理的项目表现得很差。
An Investigation into the Correlations between Students' Written Responses to Lecture-Tutorial Questions and their Understandings of Key Astrophysics Concepts