Historically, the implementation of research-based assessments (RBAs) has been a driver of educational change within physics and helped motivate adoption of interactive engagement pedagogies. Until recently, RBAs were given to students exclusively on paper and in-class; however, this approach has important drawbacks including decentralized data collection and the need to sacrifice class time. Recently, some RBAs have been moved to online platforms to address these limitations. Yet, online RBAs present new concerns such as student participation rates, test security, and students' use of outside resources. Here, we report on a study addressing these concerns in both upper-division and lower-division undergraduate physics courses. We gave RBAs to courses at five institutions; the RBAs were hosted online and featured embedded JavaScript code which collected information on students' behaviors (e.g., copying text, printing). With these data, we examine the prevalence of these behaviors, and their correlation with students' scores, to determine if online and paper-based RBAs are comparable. We find that browser loss of focus is the most common online behavior while copying and printing events were rarer. We found that correlations between these behaviors and student performance varied significantly between introductory and upper-division student populations, particularly with respect to the impact of students copying text in order to utilize internet resources. However, while the majority of students engaged in one or more of the targeted online behaviors, we found that, for our sample, none of these behaviors resulted in a significant change in the population's average performance that would threaten our ability to interpret this performance or compare it to paper-based implementations of the RBA.
We discuss the effect of administering conceptual and quantitative isomorphic problem pairs (CQIPP) back to back vs. asking students to solve only one of the problems in the CQIPP in introductory physics courses. Students who answered both questions in a CQIPP often performed better on the conceptual questions than those who answered the corresponding conceptual questions only. Although students often took advantage of the quantitative counterpart to answer a conceptual question of a CQIPP correctly, when only given the conceptual question, students seldom tried to convert it into a quantitati…
A Framework for Understanding the Impact of Integrating Conceptual and Quantitative Reasoning in a Quantum Optics Tutorial on Students' Conceptual Understanding
University of Cincinnati（辛辛那提大学）；Community College of Allegheny County（阿勒格尼县社区学院）；University of Pittsburgh（匹兹堡大学）
There is some evidence that conceptual inquiry-based introductory physics lab curricula, such as RealTime Physics, may improve students' understanding of physics concepts. Thus, these curricula may be attractive for instructors who seek to transform their physics labs to improve student learning. However, the impact of conceptual inquiry-based lab instruction on students' attitudes and beliefs about experimental physics, as measured by the E-CLASS survey, is not yet fully understood. We present data from three curricular approaches over four semesters ($n=701$). We saw no change in E-CLASS sco…
Conceptual inventory surveys are routinely used in education research to identify student learning needs and assess instructional practices. Students might not fully engage with these instruments because of the low stakes attached to them. This paper explores tests that can be used to estimate the percentage of students in a population who might not have taken such surveys seriously. These three seriousness tests are the pattern recognition test, the easy questions test, and the uncommon answers test. These three tests are applied to sets of students who were assessed either by the Force Conce…
We describe a study on the conceptual difficulties faced by college students in understanding hydrodynamics of ideal fluids. This study was based on responses obtained in hundreds of written exams and oral interviews, which were held with first-year Engineering and Science university students. Their responses allowed us to identify a series of misconceptions unreported in the literature so far. The study findings demonstrate that the most important difficulties arise from the students' inability to establish a link between the kinematics and dynamics of moving fluids, and from a lack of unders…
We discuss the development of a research-based conceptual multiple-choice survey related to magnetism. We also discuss the use of the survey to investigate gender differences in students' difficulties with concepts related to magnetism. We find that while there was no gender difference on the pre-test, female students performed significantly worse than male students when the survey was given as a post-test in traditionally taught calculus-based introductory physics courses (similar results in both the regular and honors versions of the course). In the algebra-based courses, the performance of …
Student learning in upper division thermal physics has not been studied to the same extent as in other courses like electromagnetism and quantum mechanics. Studies addressing reasoning and learning at the graduate level are even more limited. In this study, we conducted think-aloud interviews with eight graduate students involving questions centered around a set of entropy related conceptual tasks, two of which are similar to tasks presented to undergraduates in other studies. We discuss patterns in student reasoning on each question then discuss themes that appeared across questions. We ident…
This paper presents a case-study assessing and analyzing student engagement with and responses to binary, descriptive, and computational questions testing the concepts underlying resistor networks (series and parallel combinations). The participants of the study were undergraduate students enrolled in a university in Pakistan. The majority of students struggled with the descriptive question, even when successfully answering the binary and computational ones, failed to build an expectation for the answer, and betrayed significant lack of conceptual understanding in the process. The data collect…
We examine the difficulties that introductory physics students, undergraduate physics majors, and physics graduate students have with concepts related to conductors and insulators covered in introductory physics by giving written tests and interviewing a subset of students. We find that even graduate students have serious difficulties with these concepts. We develop tutorials related to these topics and evaluate their effectiveness by comparing the performance on written pre-/post-tests and interviews of students who received traditional instruction vs. those who learned using tutorials.
We investigate student understanding of energy and momentum concepts at the level of introductory physics by designing and administering a 25-item multiple choice test and conducting individual interviews. We find that most students have difficulty in qualitatively interpreting basic principles related to energy and momentum and in applying them in physical situations. The test development process and a summary of results are presented.
We analyze the effectiveness of working in pairs on the Conceptual Survey of Electricity and Magnetism test in a calculus-based introductory physics course. Students who collaborated with a peer showed significantly larger normalized gain on individual testing than those who did not collaborate. We did not find statistically significant differences between the performance of students who were given an opportunity to formulate their own response before the peer discussions, compared to those who were not. Peer collaboration also shows evidence for co-construction of knowledge. Discussions with …
We discuss the development and validation of a conceptual multiple-choice survey instrument called the Survey of Thermodynamic Processes and First and Second Laws (STPFaSL) suitable for introductory physics courses. The survey instrument uses common student difficulties with these concepts as resources in that the incorrect answers to the multiple-choice questions were guided by them. After the development and validation of the survey instrument, the final version was administered at six different institutions. It was administered to introductory physics students in various traditionally taugh…
We discuss research involving the Conceptual Survey of Electricity and Magnetism (CSEM) to evaluate one aspect of the pedagogical content knowledge of teaching assistants (TAs): the knowledge of introductory students' alternate conceptions in electricity and magnetism as revealed by the CSEM. For each item on the CSEM, the TAs were asked to (1) identify the most common incorrect answer choice of introductory physics students and (2) predict the percentage of introductory students who would answer the question correctly in a post-test. Then, we used the CSEM post-test data from approximately 40…
In this companion paper, we explore the use of isomorphic problem pairs (IPPs) to assess introductory physics students' ability to solve and successfully transfer problem-solving knowledge from one context to another in mechanics. We call the paired problems "isomorphic" because they require the same physics principle to solve them. We analyze written responses and individual discussions to a range of isomorphic problems. We examine potential factors that may help or hinder transfer of problem-solving skills from one problem in a pair to the other. For some paired isomorphic problems, one cont…
The research reported in this article represents a systematic, multi-year investigation of student understanding of the behavior of bipolar junction transistor circuits using a variety of different tasks to isolate and probe key aspects of transistor circuit behavior. The participants in this study were undergraduates enrolled in upper-division physics electronics courses at three institutions, as well as undergraduates in upper-division engineering electronics courses at one of the institutions. Findings from this research indicate that many students have not developed a robust conceptual und…
Measuring student understanding in quantum computing: Development and validation of the Quantum Computing Conceptual Survey
California State University San Marcos（圣马科斯加州州立大学）；George Mason University（乔治梅森大学）；University of Colorado Boulder（科罗拉多大学博尔德分校）；California State University Fullerton（富尔顿加州州立大学）
Ishimoto, Davenport, and Wittmann have previously reported analyses of data from student responses to the Force and Motion Conceptual Evaluation (FMCE), in which they used item response curves (IRCs) to make claims about American and Japanese students' relative likelihood to choose certain incorrect responses to some questions. We have used an independent data set of over 6,500 American students' responses to the FMCE to generate IRCs to test their claims. Converting the IRCs to vectors, we used dot product analysis to compare each response item quantitatively. For most questions, our analyses…
Exploring Generative AI assisted feedback writing for students' written responses to a physics conceptual question with prompt engineering and few-shot learning
Transfer of learning is an important objective of education. However, students usually have difficulties in solving physics transfer tasks even after having solved similar problems previously. We investigated if instruction provided using videos containing detailed explanations of previously solved problems will improve students' performance in tackling near and far transfer tasks. We also investigated whether the combination of visual cues followed by video solutions yields further enhancement of students' performance. N=33 students in an algebra-based physics class participated in an intervi…
The cyclic format of the undergraduate physics curriculum depends on students' ability to recall and utilize material covered in prior courses in order to reliably build on that knowledge in later courses. However, there is evidence to suggest that people often do not retain all, or even most, of what they learned previously. How much information is retained appears to be dependent both on the individuals' approach to learning as well as the style of instruction. In particular, there is evidence to suggest that active engagement techniques in the classroom can improve students' retention of th…
As part of ongoing investigations into student learning in advanced undergraduate courses, we have developed a conceptual assessment tool for upper-division electrodynamics (E&M II): the Colorado UppeR-division ElectrodyNamics Test (CURrENT). This is a free response, post-instruction diagnostic with 6 multi-part questions, an optional 3-question pre-instruction test, and accompanying grading rubrics. The instrument's development was guided by faculty-consensus learning goals and research into common student difficulties. It can be used to gauge the effectiveness of transformed pedagogy, and to…
Instruction is unlikely to be effective if instructors do not know the common alternate conceptions of introductory physics students and explicitly take into account common student difficulties in their instructional design. Here, we discuss research involving the Conceptual Survey of Electricity and Magnetism (CSEM) to evaluate one aspect of the pedagogical content knowledge of teaching assistants (TAs): knowledge of introductory students' alternate conceptions in electricity and magnetism as revealed by the CSEM. For each item on the CSEM, the TAs were asked to identify the most common incor…
Historically, the implementation of research-based assessments (RBAs) has been a driver of education change within physics and helped motivate adoption of interactive engagement pedagogies. Until recently, RBAs were given to students exclusively on paper and in-class; however, this approach has important drawbacks including decentralized data collection and the need to sacrifice class time. Recently, some RBAs have been moved to online platforms to address these limitations. Yet, online RBAs present new concerns such as student participation rates, test security, and students' use of outside r…
Newton's third law is one of the most important concepts learned early in introductory mechanics courses; however, ample studies have documented a wide range of students' misconceptions and fragmented understandings of this concept that are difficult to change through traditional instruction. This research develops a conceptual framework model to investigate students' understanding of Newton's third law through the knowledge integration perspective. The conceptual framework is established with a central idea emphasizing forces as the outcomes of an interaction instead of using the common actio…
The American Physical Society calls on its members to improve the diversity of physics by supporting an inclusive culture that encourages women and Black, Indigenous, and people of color to become physicists. In the current educational system, it is unlikely for a student to become a physicist if they do not share the same attitudes about what it means to learn and do physics as those held by most professional physicists. Evidence shows college physics courses and degree programs do not support students in developing these attitudes. Rather physics education filters out students who do not ent…
Research suggests that students often have difficulty transferring their learning from one context to another. We examine upper-level undergraduate and graduate students' facility with questions about the interference pattern in the double-slit experiment (DSE) with single photons and polarizers of various orientations placed in front of one or both slits. Before answering these types of questions, students had worked through a tutorial on the Mach-Zehnder Interferometer (MZI) in which they learned about interference of single photons when polarizers of various orientations are placed in the t…
Developing students' ability to troubleshoot is an important learning outcome for many undergraduate physics lab courses, especially electronics courses. In other work, metacognition has been identified as an important feature of troubleshooting. However, that work has focused primarily on individual students' metacognitive processes or troubleshooting abilities. In contrast, electronics courses often require students to work in pairs, and hence students' in-class experiences likely have significant social dimensions that are not well understood. In this work, we use an existing framework for …
