Developing and validating a conceptual survey to assess introductory physics students' understanding of magnetism

Questions 1–3 are related to magnetic force on bar magnets. A common distracter in Question 1 was option (e). Some introductory physics students claimed that the net force on the middle bar magnet should be zero because the forces due to the bar magnets on the sides should cancel out. Interviews suggest that when some students saw that there is a magnet on each side of a magnet they concluded that the forces due to the magnets are opposite to each other and therefore there is no net force on the middle bar magnet. These students often did not draw a free body diagram to determine the directions for forces due to the individual bar magnets. From the students' written responses and interviews it appears that many students did not consider Newton's Third Law when they answered the questions. In Question 2, whether there is a third bar magnet or not, due to Newton's Third Law, the force magnet 1 exerts on magnet 2 should be equal in magnitude to the force magnet 2 exerts on magnet 1. However, the appearance of the third bar magnet made many students think that the force with which magnet 1 repels magnet 2 is half or twice the force with which magnet 2 repels magnet 1. On Question 3, the most common incorrect choice was option (b). Interviews confirm that because it is mentioned in the problem that magnet 1 is twice as strong as magnet 2, some students intuitively thought that the stronger the bar magnet, the larger the force due to that bar magnet.

Questions 4–6 exemplified students' difficulties in distinguishing between the north and south poles and localized positive and negative charges or between an electric dipole and a magnetic dipole. When considering these types of questions, many introductory physics students claimed that a bar magnet is an electric dipole with positive and negative charges located at the north and south poles. Thus, they incorrectly believed that a static bar magnet can exert a force on static charges or conductors. In Question 4, some interviewed students considered the north pole of the bar magnet as consisting of positive charges, which can induce negative charges on the conductor. Hence, they claimed that the bar magnet exerts a force on the conductor toward the magnet. In Question 5, students had a similar difficulty. On the post-test, more than 40% of the students chose distracter option (e). Interviews suggest that many considered the north-pole as a positive charge which repels another positive charge. This type of confusion was even more prevalent in Question 6. On the post-test, 38% of the students in calculus-based courses answered Question 6 correctly, which is not significantly different from the pre-test performance of 22% (before instruction). A common incorrect response was option (e), i.e., there is no net force on charge q₀. Both written explanations and interviews suggest that students often considered a bar magnet as an electric dipole. When the interviewer asked some students whether a magnet will retain its magnetic properties when broken into two pieces by cutting it at the center, most of them claimed that the magnetic properties will be retained, and each of the two smaller pieces will become a magnet with opposite poles at the ends. When the interviewer asked how that is possible if they had earlier claimed that the north and south poles are essentially localized opposite charges, some students admitted they were unsure about how to explain the development of opposite poles at each end of the smaller magnets but others provided creative explanations about how charges will move around while the magnet is being cut into two pieces to ensure that each of the cut pieces has a north and south pole. During the interviews, we explicitly probed how students developed the misconception that a bar magnet must have opposite charges localized at the two poles because when a child plays with a bar magnet, he/she does not necessarily think about localized electric charges at the two ends of the magnet. Some interviewed students claimed that they learned it from an adult while he/she tried to explain why a bar magnet behaves the way it does or they had heard it on a television program. More research is needed to understand how and at what age college students developed such incorrect notions.

Questions 7 and 8 assess students' understanding of magnetic field inside or outside a bar magnet. In Question 7, the most common distractor was option (e). Students often asserted that the magnetic field at the midpoint between the north and south poles should definitely be zero. Some of them even believed that there should be no magnetic field inside of the bar magnet. In addition, students' responses to Question 8 revealed another misconception. They claimed that the magnetic field should be zero not only at the midpoint between the poles, but also at any points on the perpendicular bisector of the bar magnet including those points outside of the magnet.

Students' answers to some MCS test questions suggest that when they were given the directions of two of the quantities out of velocity of the particle, magnetic field and magnetic force, they had difficulty in determining the direction of the third quantity. Tables A1 and A2 in the appendix show that student performance on the post-test was comparable to that on the pre-test on Question 14. The most common incorrect choice for Question 14 was option (b). Interviews suggest that students often used the irrelevant information about the angle provided and had difficulty visualizing the situation in three-dimensions. The correct answer is option (e) because the velocity of all of the three charged particles is perpendicular to the magnetic field. Written explanations and interviews suggest that some students incorrectly used the superfluous information provided about the angles that the charged particles (1) and (3) make with the reference. During interviews, only when some of the students choosing option (b) were asked explicit questions about the direction of the magnetic field, velocity and the angle between them for particles (1) and (3), did they realize that the angle between them is 90° in all three cases. The fact that these students initially claimed that the angle between the magnetic field and velocity vector is 45° for particles (1) and (3) is similar to our finding in mechanics [50]. In the context of kinematics, when introductory physics students were asked a question about a rabbit's motion whose ears were at 45° angle to the horizontal, approximately one third of the introductory students used this irrelevant information about the angle of the rabbit's ears to solve the problem incorrectly. On Question 17, the performance of students in the calculus-based courses changed from 14% on the pre-test to 44% on the post-test. It is similar to the performance of students in algebra-based courses. The most common incorrect response to Question 17 was option (a) (but options (b) and (c) were also frequently selected). In written explanations, students who chose option (a) often incorrectly claimed that the velocity of a charged particle and the magnetic force can be at any angle to each other (but some correctly claimed that the magnetic field and force must be perpendicular while others incorrectly thought that the field and force must be parallel). In interviews, students who provided incorrect responses were asked to write down an expression for the magnetic force on the charged particle in an external magnetic field. Approximately half of the students were unable to write the correct expression. Those who wrote the correct expression were explicitly asked about the cross product between the velocity and magnetic field and what it implies about the angle between the magnetic force and the velocity or the magnetic field vector. Approximately half of these students corrected their initial error and claimed that the cross product implies that the magnetic force must be perpendicular to both the velocity of the charge and the magnetic field. Others did not know that the cross product of two vectors must be perpendicular to each of the vectors. In Question 10, a common distracter was option (e) which suggests that students have difficulty in realizing that for electrons, the direction of magnetic force should be opposite to the direction for positive charges (as given by the right hand rule). This is also one of the difficulties students displayed on Question 20. On Question 20, students must first use the relation between the radius of the trajectory and the mass of a charged particle to determine whether the particle whose motion is described in a magnetic field is an electron or proton. For Question 20, all incorrect options were popular. Interview and written explanations suggest that students have difficulty in deducing the relation between the radius of trajectory and the mass of a charged particle in a magnetic field. There were also some students who correctly found that the particle is an electron. However, they often incorrectly selected option (d) because of a difficulty similar to that discussed with response to Question 10 earlier.

Questions 11 and 12 are related to the work done by the magnetic field. They also probe whether students understand that the magnetic force is perpendicular to the velocity of a charged particle and only changes the direction of motion of the particle. On Questions 11 and 12, several distracters were popular. Some students claimed that the work done by the magnetic force is always positive while others thought that it can be positive or negative depending on the charge or the orientation of the initial velocity. Some students referred to the equation relating work to the inner product of force and displacement to solve for the work done by the magnetic force. However, they did not realize that the magnetic force is always perpendicular to the direction of motion and instead often thought that the directions of force and velocity of the charged particle can form any angle and the magnetic force can do work.

Previous research in mechanics shows that students often believe that motion always implies a force in the direction of motion [1–4]. Our findings are consistent with previous research. On Question 9, a common incorrect option was (c), i.e., there is a force due to the electron's initial velocity. Another common incorrect option was (d). In response to the Question 13, students performed significantly better on the post-test compared to pre-test. The most common incorrect response to Question 13 was option (c) followed by options (d) and (e). Moreover, interviewed students were asked such questions in the context of a lecture-demonstration related to the effect of bringing a powerful bar magnet from different angles towards an electron beam (including the case in which the magnetic field and velocity vectors are collinear as in Question 13). The interviewed students were explicitly asked to predict the outcome for both the electron beam and a beam of positive charges. Those who incorrectly selected option (c) in Question 13 explained that the particle will slow down because the magnetic field is opposite to the direction of velocity and this implies that the force on the charged particle must be opposite to the velocity. Of course, this prediction could not be verified by performing the experiment and left students in a confused state. Students who selected options (d) or (e) often incorrectly remembered the right hand rule. They justified their response by providing animated explanations such as 'Oh, the electron wants to get out of the way of the magnetic field and that's why it bends' in a magnetic field. They were surprised to observe that the deflection of the electron beam is negligible when the strong magnet has its north-pole or south-pole pointing straight at the beam.

Questions 15, 19 and 26 are related to the net force on charged particles in a magnetic field and another field (e.g., an electric or gravitational field). In these questions, students need to be able to figure out, e.g., the directions of magnetic force and electric force on a charge and then reflect upon whether the forces are comparable based on the information provided. Questions 15 and 19 are similar types of questions and the differences are in the directions of the magnetic field and electric field in each question. Question 26 is similar to Question 19 in which the electric field it is replaced with gravitational field. Students' response distributions for these questions were similar. As tables A1 and A2 in the appendix show, introductory physics students' choices are almost equally distributed. The percentages of correct responses on post-test on these questions ranged from 27% to 30% in calculus-based classes and from 24% to 28% in algebra-based classes. Interviews and written explanations suggest that there are two main reasons that students have difficulty in answering these types of questions. One reason is that students only consider the effect of one force, magnetic force or electric force (or the gravitational force). Therefore, they often conclude that the direction of net force is in the direction of the magnetic force or electric force (or gravitational force). The other reason is that students don't know how to determine the direction of net force with two forces in opposite directions. To answer the questions correctly, they should notice that they were not provided any information to evaluate the magnitude of the forces. Therefore, the direction of the net force cannot be determined. However, written explanations and interviews suggest that some students thought that the force had to be in one direction, either in the direction of magnetic force or the direction of electric force (or gravitational force). When they had difficulty in determining which of those two directions to choose, they often selected one randomly. Those students who claimed that the net force is zero often claimed that because the directions of the two forces are opposite to each other, the net force must be zero.

In Question 18, the directions of electric field and magnetic field are parallel. Many students' answers to this question reveal another aspect of their difficulties. Table A2 shows that 51% of calculus-based students provided the correct response on the post-test, three times the number on the pre-test. The most common incorrect option in response to Question 18 was option (c). Written explanations and interviews suggest that students had learned about velocity selectors in which the electric and magnetic fields are perpendicular to each other (not the situation shown here) and to the velocity of the particles and there is a particular speed v = E/B for which the net force on the particle is zero. These students had over-generalized this situation and had simply memorized that whenever the velocity is perpendicular to both the electric and magnetic fields, the net force on the charged particle is zero. They neglected to account for the fact that the magnitudes of electric and magnetic fields must satisfy v = E/B and the two fields must be perpendicular to each other and oriented appropriately for the net force on the particle to be zero. During interviews, students were often not systematic in their approach and talked about the net effect of the electric and magnetic fields simultaneously rather than drawing a free body diagram and considering the contributions of each field separately first. A systematic approach to analyzing this problem involves considering the direction and magnitude of the electric force ${\vec{F}}_{E}=q\vec{E}$ and magnetic force $\,{\vec{F}}_{B}=q\vec{v}\times \vec{B}$ individually and then taking their vector sum to find the net force. Some interviewed students who made guesses based upon their recollection of the velocity selector example discussed in the class claimed that the net force on the particle is zero for this situation. They were asked by the interviewer to draw a free body diagram for the case in which the charged particle is launched perpendicular to both fields. Some of them who knew the right hand rule for the magnetic force and the fact that the electric field and force are collinear $({\vec{F}}_{E}=q\vec{E})$ were able to draw correct diagrams showing that the electric force and magnetic force are not even collinear. One of these students exclaimed: 'I do not know what I was thinking when I said that the net force is zero in this case. These two are (pointing at the electric and magnetic forces in the free body diagram he drew) perpendicular and can never cancel out'. Such discussions with students suggest the need to help them learn a systematic approach to solving problems so that they do not treat a conceptual problem (such as those on MCS) as a guessing task [1–4].

Questions 25 and 27 are related to the net force of one on two parallel current carrying wires in a magnetic field. Students' responses to these two problems reflect the difficulties that are similar to those shown in Questions 15, 19 and 26. Question 25 was the most difficult question in the test. On post-test, only 21% of the calculus-based students responded correctly. On the post-test, options (b) and (c) were the most common incorrect distracters. Interviews suggest that students answered this question incorrectly either because they only considered the effect of one force or because they randomly chose a direction for the force since they were unable to compare the magnitude of the forces. Unlike previous problems asking for the direction of net force, Question 27 asks students to predict the possible direction of the magnetic field to make the net force zero. Student performance on Question 27 was better than Question 25 and all distracters were popular.

We observed that when students were presented with parallel current carrying wires with current flowing into or out of the page, they often failed to reason about the magnetic field or magnetic force due to the wires by using the right hand rule. Students' responses to Questions 22–24 reflect this difficulty. Question 22 is about the direction of the force on a current carrying wire due to a parallel wire carrying current in the opposite direction. The most common incorrect response (option (d)) was chosen based on the incorrect reasoning that the wires would attract each other. Even in the individual lecture-demonstration based interviews, many students explicitly predicted that the wires would attract and come closer to each other. When asked to explain their reasoning, students often cited the maxim 'opposites attract' and frequently made an explicit analogy between two opposite charges attracting each other and two wires carrying current in opposite directions attracting each other. In lecture-demonstration based interviews, when students performed the experiment and observed the repulsion, none of them could reconcile the differences between their initial prediction and observation. One difficulty with Question 22 is that there are several distinct steps (as opposed to simply one step) involved in reasoning correctly to arrive at the correct response. In particular, understanding the repulsion between two wires carrying current in opposite directions requires comprehension of the following issues: (I) there is a magnetic field produced by each wire at the location of the other wire. (II) The direction of the magnetic field produced by each current carrying wire is given by a right hand rule. (III) The magnetic field produced by one wire will act as an external magnetic field for the other wire and will lead to a force on the other wire whose direction is given by a right hand rule. None of the interviewed students was able to explain the reasoning systematically. What is equally interesting is the fact that none of the interviewed students who correctly predicted that the wires carrying current in opposite directions would repel could explain this observation based upon the force on a current carrying wire in a magnetic field even when explicitly asked to do so. A majority of them appeared to have memorized that wires carrying current in opposite directions repel. They had difficulty applying both right hand rules systematically to explain their reasoning: the one about the magnetic field produced by wire 2 at the location of wire 1 and the other one about the force on wire 1 due to the magnetic field. This difficulty in explaining their prediction points to the importance of asking students to explain their reasoning. On Question 24, the most common incorrect response was option (c). Similar to Question 22, students often did not follow the steps mentioned above to figure out the directions of magnetic forces and simply used 'opposites attract'. As can be seen from tables A1 and A2, for Question 23, option (e) was the most common incorrect response on both the pre-test and post-test. These students claimed that the magnetic field is zero at point P between the wires. In written explanations and individual interviews, students who claimed that the magnetic field is zero at point P between the wires carrying opposite currents argued that at point P, the two wires will produce equal magnitude magnetic fields pointing in opposite directions, which will cancel out. During interviews, when students were explicitly asked to explain how to find the direction of the magnetic field at point P, they had difficulty figuring out the direction of the magnetic field produced by a current carrying wire using the right hand rule and in using the superposition principle to conclude correctly that the magnetic field at point P is upward. Interviews suggest that for some students, confusion about analogous concepts in electrostatics was never cleared and could have exacerbated to difficulties related to magnetism. For example, several students drew explicit analogy with the electric field between two charges of equal magnitude but opposite sign, incorrectly claiming that the electric field at the midpoint should be zero because the contributions to the electric field due to the two charges cancel out similar to the magnetic field canceling out in Question 23. Some interviewed students who were asked to justify their claim that the electric field at the midpoint between two equal magnitude charges with opposite sign is zero were reluctant, claiming the result was obvious. Then, the interviewer told them that simply stating that the influence of equal and opposite charges at the midpoint between them must cancel out is not a good explanation and they must explicitly show the direction of the electric field due to each charge and then find the net field. This process turned out to be impossible for some of them but others who drew the electric field due to each charge in the same direction at the midpoint were surprised. One of them who realized that the electric field cannot be zero at the midpoint between the two equal and opposite charges smiled and noted that this fact is so amazing that he will think carefully later on about why the field did not cancel out at the midpoint. There is a question similar to Question 23 on the broad survey of electricity and magnetism CSEM [11]. However, students performed much worse on Question 23 than on the corresponding question on the CSEM (50% for calculus-based courses on post-test for Question 23 versus 63% for the corresponding question on the CSEM). One major difference is that we asked students for the direction of the magnetic field at the midpoint between current carrying wires carrying current in the opposite directions (whereas CSEM asked for the direction of the magnetic field at a point which was not in the middle), and many students had the misconception that the magnetic field is zero at the midpoint. Thus, they gravitated to option (e). In the CSEM, all incorrect choices were quite popular because the misconception specifically targeted by Question 23 about the magnetic field being zero at the midpoint between the wires was not targeted there.

Questions 28–30 investigated student difficulties in determining the magnetic field generated by a current loop. On post-test, the most common misconception amongst calculus-based students for Question 28 was that the magnetic field inside a loop should be zero. Interviews suggest that students confused the magnetic field inside a 2D loop with the electric field inside a 3D sphere with uniform charge. For the same students, on Question 29, the most common incorrect option was option (e) followed by option (a). Many students thought that the magnetic field outside of a loop should be zero or in the same direction of the field inside the loop. On Question 30, the most common distracter was option (a). Interviews and written explanations suggest that, similar to determining the direction of the magnetic field due to two current carrying wires pointing into/out of the page, students were likely to use 'opposites attract' to solve the problem. Some interviewed students incorrectly claimed that two current loops with current flowing in the opposite directions should attract each other, therefore, the magnetic field in between the two loops for that case should add up. These responses based upon naïve intuition suggest that students should be provided guidance to follow systematic approaches to solve even conceptual problems [1].

The performance of many students was closely tied to their ability to apply the right hand rule and three-dimension visualization. In magnetism, there are many questions related to magnetic field and magnetic forces. The students are often given a current carrying wire or moving charge in a magnetic field and they have to determine the magnetic force. Therefore, students must know how to use the right hand rule. For example, Question 21 requires the application of the right hand rule once. However, the results shown in tables A1 and A2 suggest that many students had difficulty with it. In response to Question 21, many students incorrectly chose either the direction opposite to the correct direction or the direction of the current.

The requirement of 3D visualization is also high in magnetism because the directions of magnetic force is always perpendicular to the magnetic field and the direction of motion of the charged particle. In the MCS test, Questions 10–30 all require the ability to visualize in three-dimensions. Question 16 is a good example involving 3D visualization. In this problem, students need to understand that the proton will undergo circular motion in the plane that is perpendicular to the page and simultaneously move along the direction of magnetic field with a constant speed making its overall path helical. Failure to decompose the initial velocity into two components and use the correct sub-velocity to figure out the magnetic force results in incorrect responses.

We also administered the MCS test as a pre-test and post-test to the upper level undergraduate physics majors enrolled in an E&M course which used David Griffith's E&M textbook (typically, taken in second or third year in college). 26 upper-level physics majors took the pretest and 25 took the post-test. Tables A3 and A4 in the appendix show the percentage of upper-level physics majors who made the various choices on the MCS questions on the pre-test and post-test, respectively. The average pre-test and post-test scores are 51% and 59%, respectively and this difference in the average scores from pre-test to post-test is not statistically significant. Also, the pre-/post-test scores of upper-level physics majors is comparable to the post-test score of the introductory students in the calculus-based courses (discussed earlier). In other words, it appears that the upper-level physics majors did not have a better conceptual understanding of magnetism than students in calculus-based introductory physics course (taken in the first year primarily by physics, engineering, mathematics and chemistry majors).

To benchmark the MCS test, we also administered it over two consecutive years to a total of 42 first year physics PhD students who were enrolled in a professional development course for the teaching assistants (TAs). These PhD students were simultaneously enrolled in the first semester of the graduate E&M course. The PhD students were told ahead of time that they would be taking a test related to magnetism concepts. They were asked to take the test seriously, but the actual score on the MCS test did not count for their course grade. The average test score for the PhD students was approximately 83% with the reliability coefficient KR-20 of 0.87 (which is excellent [10]). The better performance of PhD students compared to the undergraduates is statistically significant. The minimum score obtained by a PhD student was 33% and the maximum score obtained was 100% which shows that there is enormous variability in the prior preparation of the first year PhD students. Table A5 in the appendix shows the average percentage of PhD students who selected the various choices on each question of the MCS test.