Characterizing and monitoring student discomfort in upper-division quantum mechanics

We investigate student comfort with the material in an upper-division spins-first quantum mechanics course. Pre-lecture surveys probing students' comfort were administered weekly, in which students assigned themselves a"discomfort level"on a scale of 0--10 and provided a written explanation for their choice. The weekly class-wide average discomfort level was effectively constant over the semester, suggesting that the class found no single unit especially jarring nor especially easy. Student written responses were coded according to their reported source of discomfort---math, math-physics connection, physics, and notation. The relative prevalence of these categories varied significantly over the semester, indicating that students find that different units present different challenges, and also that some of these challenges fade in importance as the semester progresses. Semi-structured interviews with students in a similar quantum mechanics course at a different institution provided additional context and insight into these results.


I. INTRODUCTION
Upper-division quantum mechanics (QM) is known to be both conceptually and mathematically challenging for students 1 . Much research has focused on student difficulties, but recent work has also investigated student perceptions of the course-e.g., examining students' ideas about the nature of quantum mechanics-as a way to guide instruction and future studies 1,2 . Johansson discusses students' perceptions of their QM courses in light of cultural expectations, arguing that an expectancy mismatch can affect how students identify as physicists 3 . Schermerhorn et al. investigated whether students considered the physics or math more challenging when studying spin systems or spatial wave functions 4 . Outside of the QM context, Gupta et al. explored connections between students' emotions and conceptual reasoning 5 .
We investigate students' self-reported sense of comfort with the material in QM. Prompting students to reflect on their own understanding of recently covered material and encouraging them to identify challenging aspects has precedent in the Just-in-Time-Teaching paradigm 6 . In this paper, we present the results of a survey administered on a weekly basis in an upper-division spins-first QM course, as well as complementary preliminary results from an interview study with students from a similar course at a different university. On the survey, students were asked to identify their level of discomfort with the material along with reasons for that discomfort. In the interviews, students were prompted to reflect on the content they were learning and their sense of comfort with it.
A spins-first approach is one of two common paradigms for upper-division QM instruction 7 . Where a positionsfirst approach focuses on solving the Schrödinger equation for continuous wave functions that describe the position-or momentum-space distribution of a particle, a spins-first approach instead begins with spin-½ systems described by a discrete, two-state basis before a transition is made to studying continuum systems near the end of the semester. The spinsfirst structure emphasizes fundamental quantum mechanical concepts in part by postponing development of the more computationally intensive mathematical formalism required for continuous wave functions 7 . However, one concern is that students may find the discrete-to-continuous transition especially difficult or jarring 8 .
A marked increase in student discomfort following the course's switch to studying spatial wave functions would highlight the aforementioned concern about the difficulty of this transition point. More generally, the degree of and reasons for student discomfort with the material illuminate the things that students consider important in their own learning, which can inform instruction as well as future research.

II. METHODOLOGY
This paper focuses on student responses to a weekly prelecture survey administered in one semester of upper-division QM at University A, a large, public, primarily undergraduate and Hispanic-serving institution. Preliminary analysis of these data also prompted an interview study with students in the first semester upper-division QM course at University B, an R1, PhD-granting institution. The interview study was developed during and ran concurrently with continued analysis of the survey data. Both courses adhered to a "spins-first" QM curriculum using McIntyre's text 9 . The course at University A was taught by physics education research faculty.
A. Pre-lecture surveys A pre-lecture assignment was administered online before each class to students in an (in-person) upper-division QM course at University A, where a total of 26 students responded at least once. Students received participation credit for completing the survey, but it was not graded for correctness. Beginning in the semester's third week, the first survey of each week opened with the same two questions asking students to reflect on their comfort with the material they were learning.
Students were asked to rank their discomfort level on a scale of 0-10 as follows: On a scale of 0-10, rate your discomfort (or comfort) with the ideas presented in class this week.
(We are not asking about how well you think you can answer homework or exam questions, but how the concepts and ideas are "sitting" with you and your intuition about the world.) 0 -(no discomfort: "all these ideas make complete sense and seem reasonable") 5 -(moderate discomfort: "some of the ideas seem illogical and bother me, but I can see what's going on") 10 -(complete discomfort: "none of these ideas make any sense and I'm deeply concerned") Think of this as a "quantum pain scale." Students were then asked to explain their discomfort level: If you selected a number greater than 0, give a concrete example of an idea or concept that makes you uncomfortable.
Not all students provided explanations on every survey, but usually they wrote one or two sentences.
We examined weekly and individual-student averages of the numerical responses. The written responses were categorized using an emergent coding strategy to identify the type of material the student reported discomfort with (see Sec. III B).
Once the coding scheme was agreed upon, codes were assigned to all responses by two researchers working independently, with 75% initial agreement. Mismatches were resolved in discussion between the two researchers until 100% agreement was reached. Individual responses were assigned multiple codes if the student reported multiple sources of discomfort, meaning that some disagreements were resolved by adopting the codes suggested by both researchers.

B. Semi-structured interviews
In the semester after the pre-lecture survey data were collected, an interview study was performed to probe the reasons for student discomfort in more detail. Interviewees were all drawn from the first semester upper-division QM course at University B. The same six volunteers were interviewed individually several times, including during the time evolution unit in weeks 6-7, and immediately after the infinite square well unit-the first unit in the wave functions context-in week 10. The volunteers were compensated for participation.
The interviews were conducted in spring 2020, and the second round took place one week after University B transitioned to remote learning due to COVID-19. Instruction on the infinite square well unit preceded the switch. (The pre-lecture surveys were administered in an unaffected semester.) The interview protocol prompted students to share how they were feeling overall in the course, to assess their comfort level with material in general, and to reflect specifically on their comfort level with certain topics, including notation, math, and physics (i.e., physical concepts). The protocol also included additional questions outside the scope of this paper. Analysis of the interview data is ongoing, but preliminary results are presented in Sec. IV.

III. RESULTS
In this section, we present student responses to both questions on the pre-lecture surveys. First, we present the average selfreported discomfort level on individual-student and weekly bases. Then we explore qualitative analysis of students' written explanations for their reported discomfort level.
A. Tracking discomfort level over the semester Figure 1 presents the distribution of students' discomfort levels averaged over the entire semester. Centered around the overall average of 4, the distribution is reasonably normal. Individual student responses changed on a week-by-week basis by ±1.5 on average. The figure shows some correlation between higher discomfort score and poorer course performance, but it is unclear if this reflects students' ability to diagnose their own difficulties, their emotional response to the grades they received on assignments, or another effect.
This quantitative discomfort scale is, at best, calibrated on an individual-student basis. That is, we hope that a 4/10 discomfort level means approximately the same thing to the same student each week, but we cannot assume it means the Number of students

Self-reported discomfort level individual-student average
Course Grade Upper 50% Lower 50% FIG. 1. The histogram shows the distribution of students' average self-reported discomfort levels. Averages are calculated over all of the pre-lecture surveys that the given student responded to. The individual-student standard deviation averaged over all students is 1.5. The outlying student on the right only reported their discomfort level on one survey, and the one on the left exclusively reported zeroes. The darker (lighter) colored bars indicate students whose final course grade fell in the upper (lower) 50% of the class.
same thing to every student. This makes it difficult to compare the magnitude of discomfort level across students. Still, averaging over students may capture class-wide shifts in discomfort. Figure 2 shows the class-wide average discomfort level reported on each of the 12 weekly pre-lecture surveys administered over the semester. The class-wide average discomfort level is effectively constant over the entire semester: it hovers close to 4 and varies by less than 1. Additionally, there is a sizable spread every week: the standard deviation is usually 2 or higher. Each week, some students report a high level of discomfort while others report a low level, but there is never a net trend towards higher or lower class-wide discomfort, regardless of changes in the topical areas being covered.
B. Identifying categories of discomfort and tracking their prevalence over the semester Qualitative analysis focused on students' explanations for their discomfort level on the pre-lecture surveys. Through emergent coding, we identified four categories of discomfort corresponding with different aspects of the material: math, math-physics connection, physics, and notation. We also employed a fallback category, other. We define each of these categories as follows. The percentages indicate the frequency of the given category amongst the 241 total assigned codes.
Math (30%): The response references either conceptual or procedural understanding of the mathematics, such as discomfort computing integrals. For example, "The concepts weren't too difficult I think I just need to practice the calculations . . . ." Math-Physics (16%): The response references the connection between the math and the physics, such as discomfort with the physical meaning of an equation. For example, "I'm mostly uncomfortable with knowing how and when to use the equations." Physics (24%): The response references physical concepts, such as questions about measurement. For example, "I am confused but intrigued by entanglement . . . ." This category also includes responses that reference specific content but are too vague for another category, such as, "I was just having some problems with the clicker questions for measurement and measuring uncertainty . . . ." Notation (10%): The response references notation used in the course, usually Dirac notation. For example, "The Dirac notation and braket rules are still fuzzy . . . ." Other (20%): The response does not reference specific content covered in the course. Examples include, "Exams are coming and i am having anxiety . . . ," and "Still getting the hang of things." We focus on the four content-centered categories in this work.
Overall, more students cite discomfort with math than any other category, with physics being the second-most common, followed by math-physics and notation. However, this ranking was not consistent on a week-by-week basis. Figure 3 presents the distribution of discomfort categories for each week that the pre-lecture survey was administered.
Early on, math and notation are the dominant sources of discomfort. While students report discomfort with math Each cross-section is normalized to the total number of codes assigned to student responses for the given week-this number is included in the labels on the horizontal axis. The topics in the labels are the same as in Fig. 2. Some student responses were assigned multiple codes, and not all students provided a written explanation for their reported discomfort level.
throughout the semester, the notation category all but disappears by the fourth pre-lecture. Meanwhile, students report increasing discomfort with physical concepts as the semester progresses, peaking during the units on quantum computing and entanglement, where the physics category dominates. The physics category drops back to below 20% during the "Infinite Square Well" unit, which is when the course switches to discussing continuous wave functions as opposed to discrete spin systems. We also see that math-physics is the dominant source of discomfort during this unit, accounting for about 40% of all reported discomfort, more than twice its maximum in any other week. We also see a small resurgence of the notation category at this time, with two students referencing notation as a source of discomfort in either this unit or the following one. Finally, we note that the other category surpasses 20% only in the one or two weeks preceding each exam.
Most individual students' responses included a diversity of categories over the semester. One student did not provide a single written response. Of the remaining students, all 25 reported discomfort with math at least once, 24/25 with physics, and 19/25 with math-physics, although notation only arose for 15/25 students. Only 10/25 students reported discomfort with any single category more than 50% of the time (3 of whom favored other).
To investigate the relationship between students' discomfort level and categories, we generated a version of Fig. 3 wherein each occurrence of each category was weighted by the discomfort level reported by the given student in the given week. This did not produce a meaningful change, indicating that there was minimal correlation between discomfort category and discomfort level.

IV. DISCUSSION & INTERVIEW OUTCOMES
The transition from discussing discrete spin systems to continuous position-space wave functions did not produce any shift in class-wide discomfort at University A, contrary to our expectations 8 . Although conducted with a different set of students at a different school, several potential explanations arose in the second round of interviews held at University B, which immediately followed this transition point in a similarly structured QM course. Discussing the material in the wave functions unit, one student remarked, It's starting to look a little more familiar to what I've done before because we've started going over, like finite and infinite square wells. We've also started looking at waves which we're also doing in E&M.
Students have often seen examples of position-space systems like the infinite square well in previous courses, and the mathematics of waves may also feel familiar from other courses. Another student said of the wave functions unit that, compared with earlier material, she did "better with the conceptual stuff just because for me it's-it feels more applied." Other interviewees echoed this sentiment: the familiar physical meaning of "position" may help students make sense of the new system despite new mathematical formalism.
Based on qualitative data, however, the transition point still warrants instructor attention. One interviewee remarked, "I feel like the material got really hard last week, very quickly," and others reported that it took them time to find connections between the two halves of the course. The courses at both universities (A and B) included similar activities at the start of the wave functions unit designed specifically to ease the transition point, and several interviewees at University B referenced this activity as helpful 10 .
Several patterns arose in the coding of students' written explanations for their reported discomfort level. First, although Dirac notation is a clear source of discomfort for students when it is introduced at the beginning of the semester, discomfort with notation fades quickly. Interviewed students shared a similar sentiment, and some expressed that they had come to appreciate the new notation by the end of the semester. These results suggest that the existing curriculum's approach to this topic worked well for students despite their initial discomfort with Dirac notation.
We also found that more students reported discomfort with the mathematics in the first half of the semester, despite the fact that the mathematics required in the second half tends to be more computationally intensive. Interviewed students echoed this sentiment. For example, one student said he considered the class to be "70% math, 30% physical physics understanding" in his first interview. He, along with other interviewees, pointed out that they were less familiar with linear algebra and matrix manipulation than with integration and differential equations at the start of the course.
Not all interviewees felt this way, however (see also related research in 4 ). One remarked in his first interview that, "I'm way more confident in my math abilities in this course than I was in like diff-EQ or calc 3," although he still considered problem-solving in the course to be "all math." His perspective shifted in the second interview, saying, "this class is much more about understanding your approach to the problem than the math itself." This appeared to track the survey results, where units on the uncertainty principle and time dependence saw increases in student discomfort with physical concepts and a decrease in discomfort with the mathematics compared with earlier in the semester. This trend saw its peak during the units on quantum computing and entanglement, suggesting that those topics were especially effective at drawing out physical concepts.

V. CONCLUSION
Weekly surveys probing students' comfort with the material were administered in an upper-division QM course. Students' average level of discomfort remained constant over the semester, but the reasons for that discomfort varied as the course progressed and as different topics were covered. We believe our results can guide instructors and researchers towards topical areas that students may feel deserve attention. We do not, however, consider student discomfort with the course's material inherently bad. Our results can be seen as identifying the aspects of the material that are top-of-mind for students at different points in their QM class. For example, the increase in the importance of the physics discomfort category during the quantum computing and entanglement units may be an argument for teaching these topics.
Similarly, discomfort with quantum mechanics is common among physicists and could suggest an expert-like understanding of the material. This paper previews an ongoing analysis of an interview study, which will also examine students' comfort with quantum weirdness.