Machine learning algorithms have recently been used to predict students' performance in an introductory physics class. The prediction model classified students as those likely to receive an A or B or students likely to receive a grade of C, D, F or withdraw from the class. Early prediction could better allow the direction of educational interventions and the allocation of educational resources. However, the performance metrics used in that study become unreliable when used to classify whether a student would receive an A, B or C (the ABC outcome) or if they would receive a D, F or withdraw (W) from the class (the DFW outcome) because the outcome is substantially unbalanced with between 10\% to 20\% of the students receiving a D, F, or W. This work presents techniques to adjust the prediction models and alternate model performance metrics more appropriate for unbalanced outcome variables. These techniques were applied to three samples drawn from introductory mechanics classes at two institutions ($N=7184$, $1683$, and $926$). Applying the same methods as the earlier study produced a classifier that was very inaccurate, classifying only 16\% of the DFW cases correctly; tuning the model increased the DFW classification accuracy to 43\%. Using a combination of institutional and in-class data improved DFW accuracy to 53\% by the second week of class. As in the prior study, demographic variables such as gender, underrepresented minority status, first-generation college student status, and low socioeconomic status were not important variables in the final prediction models.
We present the findings of a pilot plan of active learning implemented in introductory physics in a Chilean public university. The model is research based as it considered a literature review for adequate selection and design of activities, consistent with the levels of students' reasoning skills. The level of scientific reasoning is positively correlated to student success. By contrast to a control group of students following traditional lectures, we observed a significant reduction in failure rate for students that do not yet posses formal scientific reasoning. This profile of student being …
This proposal presents a methodology called enriched three-tier test, based on a similar test previously discussed in the literature. Ours consists of the use of justification and degrees of confidence combined with a multiple-choice test. This methodology of assessment allows the teacher to diagnose learning risks, while engaging the student in a metacognitive process. The three-tier test proved to be an efficient mechanism for identifying up to eighteen tints along the learning spectrum for a single concept, which interpretation alerts the teacher about the existence of learning risks and ot…
The introduction of active learning into physics education at the university level may be crucial for improved learning outcomes. In UNIST (Ulsan National Institute of Science and Technology), introductory physics has been redesigned to reduce the time spent on unidirectional lectures, while increasing the time devoted to classroom activities which facilitate the interaction between students and the instructors, as well as increasing the interaction between student peers. We measured student achievement and the degree of student satisfaction in order to evaluate the impact of the new learning …
Challenges in addressing student difficulties with time-development of two-state quantum systems using a multiple-choice question sequence in virtual and in-person classes
Challenges in addressing student difficulties with measurement uncertainty of two-state quantum systems using a multiple-choice question sequence in online and in-person classes
Challenges in addressing student difficulties with quantum measurement of two-state quantum systems using a multiple-choice question sequence in online and in-person classes
Challenges in addressing student difficulties with basics and change of basis for two-state quantum systems using a multiple-choice question sequence in online and in-person classes
This addendum shows that a demographic grade gap is best understood using a course deficit model rather than the more common student deficit model. Students from a concepts-first class took the same final exam as the three other classes that were offered at the same time so the grade gaps between different demographic groups from each class can be compared. It is found that students identifying as members of a racial/ethnic group underrepresented in physics had higher final exam grades than their peers from the same concepts first class rather than the usual case, lower exam grades than their …
There is a large variety of goals instructors have for laboratory courses, with different courses focusing on different subsets of goals. An often implicit, but crucial, goal is to develop students` attitudes, views, and expectations about experimental physics to align with practicing experimental physicists. The assessment of laboratory courses upon this one dimension of learning has been intensively studied in US institutions using the Colorado Learning Attitudes Science Survey for Experimental Physics (E-CLASS). However, there is no such an instrument available to use in Germany, and the in…
We report on the study of student difficulties regarding heat engine in the context of Stirling cycle within upper-division undergraduate thermal physics course. An in-class test about a Stirling engine with a regenerator was taken by three classes, and the students were asked to perform one of the most basic activities---calculate the efficiency of the heat engine. Our data suggest that quite a few students have not developed a robust conceptual understanding of basic engineering knowledge of the heat engine, including the function of the regenerator and the influence of piston movements on t…
In this meta-analysis, we synthesize the results of 24 studies using the Colorado Learning Attitudes about Science Survey (CLASS) and the Maryland Physics Expectations Survey (MPEX) to answer several questions: (1) How does physics instruction impact students' beliefs? (2) When do physics majors develop expert-like beliefs? and (3) How do students' beliefs impact their learning of physics? We report that in typical physics classes, students' beliefs deteriorate or at best stay the same. There are a few types of interventions, including an explicit focus on model-building and/or developing expe…
Peer Influence on Physics Self-Efficacy and Grades: A comparative study of students in an introductory calculus-based course who typically worked alone or in groups before and during the pandemic
We present a model for competency-based grading for calculus-based introductory physics that encourages students to obtain proficiency with all course content. By allowing students to continually improve their proficiency with skills and content throughout the semester, this formative grading system is designed to create a more flexible learning environment that better accommodates the varying schedules and needs of students. While all students show improvement in their performance following the implementation of this grading system, the largest gains were found for women and first generation …
The Force Concept Inventory (FCI) can be used as an assessment tool to measure conceptual gains in a cohort of students. The FCI uses a conceptions/"misconceptions" lens rather than a context dependent perspective, such as "knowledge-in-pieces". In this study it was given to first year students ($N=256$ students) pre- and post-mechanics lectures, at the University of Johannesburg. From these results we examine the effect of switching mid-semester from traditional classes to online classes, as imposed by the COVID-19 lockdown in South Africa. Overall results indicate no appreciable difference o…
Many Physics Education Researchers have discussed the positive correlation between students' incoming mathematics skills and performance in their physics classes. Thus, in order to strengthen their performance gains in their physics courses, professors strive to bolster students' mathematics abilities but in-class time constraints force instructors to focus their lesson plans on physics concepts rather than renewing mathematics material. For the sake of preserving in-class time, many instructors are turning to lecture delivery outside of class by way of flipped classrooms and MOOCs (Massive On…
Authentic research as a method of teaching science is gaining popularity in high schools and colleges. To make this research experience most efficient, students need adequate preparation in traditional science courses. Existing materials available for inquiry-based teaching methods are not intended to prepare a student for independent research. We discuss principles for organization of laboratory classes with this goal in mind and provide examples that we have implemented for algebra-based physics, calculus-based physics, and honors chemistry for high school students.
We have developed and implemented a low-cost peer-mentoring program that blends personal and academic support to help students achieve academic success in the introductory courses required for the Physics Major or the Dual-Degree Program in Engineering. As a result of this program, retention has increased among entering science, technology, engineering and mathematics students at Albion College as they move through the introductory classes, as shown by a 20 per cent increase in retention from first-semester to third-semester physics courses compared to years when this program was not in place.
In previous work we analyzed databases for 95 classes to show that the percent grade scale was correlated with a much higher student fail rate than the 4.0 grade scale. This paper builds on this work and investigates equity gaps occurring under both scales. By employing a "Course Deficit Model" we attribute the responsibility for closing the gaps to those who are responsible for the policies that guide the course. When comparing course grades in classes graded using the percent scale with those in courses graded using the 4.0 scale, we find that students identifying as belonging to racial or e…
With inspiration from the classic study by Chi, Feltovich, and Glaser [1], we asked introductory physics students in three introductory physics classes to categorize mechanics problems based upon similarity of solutions. To evaluate the effect of problem context on students' ability to categorize, two sets of problems were developed for categorization. Some problems in one of the problem sets that students were asked to categorize included those available from the prior study by Chi et al. Our findings, which contrast from those of Chi et al., suggest that there is a much wider distribution of…
Feedback has a powerful influence on learning, but it is also expensive to provide. In large classes, it may even be impossible for instructors to provide individualized feedback. Peer assessment has received attention lately as a way of providing personalized feedback that scales to large classes. Besides these obvious benefits, some researchers have also conjectured that students learn by peer assessing, although no studies have ever conclusively demonstrated this effect. By conducting a randomized controlled trial in an introductory statistics class, we provide evidence that peer assessment…
This paper presents the first item response theory (IRT) analysis of the national data set on introductory, general education, college-level astronomy teaching using the Light and Spectroscopy Concept Inventory (LSCI). We used the difference between students' pre- and post-instruction IRT-estimated abilities as a measure of learning gain. This analysis provides deeper insights than prior publications into both the LSCI as an instrument and into the effectiveness of teaching and learning in introductory astronomy courses. Our IRT analysis supports the classical test theory findings of prior stu…
Student belongingness is important for successful study paths, and group work forms an important part of modern university physics education. To study the group dynamics of introductory physics students at the University of Helsinki, we collected network data from seven laboratory course sections of approximately 20 students each for seven consecutive weeks. The data was collected via the SocioPatterns platform, and supplemented with students' major subject, year of study and gender. We also collected the Mechanics Baseline Test to measure physics knowledge and the Colorado Learning Attitudes …
