Student difficulties with determining expectation values in quantum mechanics

The expectation value of an observable is an important concept in quantum mechanics. However, upper-level undergraduate and graduate students in physics have both conceptual and procedural difficulties when determining the expectation value of physical observables, especially when using Dirac notation. To investigate these difficulties, we administered free-response and multiple-choice questions and conducted individual interviews with students. Here, we discuss the analysis of data on student difficulties when determining the expectation value.


I. INTRODUCTION AND BACKGROUND
Learning quantum mechanics (QM) is challenging [1][2][3][4]. Investigations of student difficulties in QM are important for developing curricula and pedagogies that help students develop a solid grasp of QM [5][6][7][8][9]. However, few prior research studies have focused on the conceptual and procedural difficulties upper-level undergraduate and graduate students have with expectation values of physical observables when using Dirac notation, a compact and convenient notation used extensively in QM.
Here we discuss an investigation of difficulties that upper-level undergraduate and graduate students have with the expectation values of observables when making use of Dirac notation in courses in which this notation was used extensively. The expectation value is the average value of an observable when measurements of that observable are made on a large number of identically prepared quantum systems. It is used frequently in QM since measurement outcomes are probabilistic rather than deterministic. For each observable , there is a Hermitian operator ̂. When the quantum system is in a state |Ψ⟩ and an observable is measured in an experiment, one obtains an eigenvalue of ̂. Therefore, the expectation value ⟨Ψ|̂|Ψ⟩ in a given quantum state |Ψ⟩ can be found by summing the probability of measuring a particular eigenvalue of ̂ multiplied by that eigenvalue over all possible measurement outcomes. Ensuring that students conceptually understand the meaning of expectation value and develop proficiency in calculating it is important.
If the states {| ⟩, = 1,2,3 … ∞} form a complete set of eigenstates of a Hermitian operator ̂ corresponding to an observable with non-degenerate discrete eigenvalues (i.e., ̂| ⟩ = | ⟩), one can find the expectation value of the observable in a generic state |Ψ〉 in terms of the eigenstates and eigenvalues of ̂ by expanding |Ψ〉 as a linear superposition of the eigenstates of the operator ̂. A generic state |Ψ〉 can be written as a linear superposition of the eigenstates of an operator ̂ with discrete eigenvalues as |Ψ〉 = ∑ | 〉⟨ |Ψ⟩ = ∑ | 〉, with = ⟨ |Ψ⟩. Here, = ⟨ |Ψ⟩ is the projection of the state |Ψ⟩ along an eigenstate | ⟩ of the operator ̂ with eigenvalue and | | 2 = |⟨ |Ψ⟩| 2 is the probability of measuring . Using this, the expectation value can be found as follows: Another approach for finding the expectation value of observable is to insert the identity operator ̂ in terms of a complete set of eigenstates of the operator ̂, i.e., ̂= ∑ | 〉 ⟨ |, into the expression for the expectation value: Moreover, if the states {| ⟩} are a complete set of eigenstates of ̂ with continuous eigenvalues (i.e., ̂| ⟩ = | ⟩) and the identity operator in terms of the eigenstates of ̂ is ̂= ∫ | ⟩ +∞ −∞ ⟨ | , using a very similar approach to that used for the case in which the eigenvalue spectrum of ̂ is discrete, the expectation value of in state |Ψ〉 in terms of the eigenstates | 〉 and eigenvalues is ⟨Ψ|̂|Ψ⟩ = ∫ +∞ −∞ |⟨ |Ψ⟩| 2 .

II. METHODOLOGY
Student difficulties were investigated by administering multiple-choice and open-ended questions to upper-level undergraduate (UG) and graduate (G) students in QM courses after traditional instruction in relevant concepts. We observed difficulties on these questions which were administered on in-class quizzes and exams. The UG students were enrolled in a junior/senior level QM course and the G students were enrolled in a first year core graduate QM course. Table I lists the questions that were administered to students as part of this investigation. The multiple-choice question was administered to 184 upper-level UG students after traditional instruction as part of a quiz at four U.S. universities (see Question Q1 in Table I). The open-ended quiz and exam questions were administered to UG and G students after traditional instruction in QM at the University of Pittsburgh over several years (see questions Q2 and Q3 in Table I). The number of students answering the open-ended questions Q2 and Q3 is different in Table I because, in some of the years, UG students were not given question Q3 and we did not include students in data analysis if they left a question blank (e.g., Q2 has 65 G and Q3 has 62 G because three graduate students left Q3 blank). Since the performance on Table I. Questions involving expectation value that were administered to students and the number of students ( ) answering the questions. The correct answer is bolded. Questions Q1. Suppose {| ⟩, = 1,2,3 … ∞} forms a complete set of orthonormal eigenstates of an operator ̂ corresponding to a physical observable with nondegenerate eigenvalues . ̂ is the identity operator. Choose all of the following statements that are correct. The open-ended questions were graded using rubrics which were developed by the two investigators together. A subset of the open-ended questions was graded separately by the investigators. After comparing the grading of the openended questions, the investigators discussed any disagreements in grading and resolved them with a final inter-rater reliability of better than 95%.
Student difficulties were also investigated by conducting individual interviews with 23 upper-level UG and G student volunteers enrolled in the QM courses (not necessarily the same students who answered the written questions). The individual interviews employed a think-aloud protocol to better understand the rationale for students' written responses. During the semi-structured interviews, we asked students to "think aloud" while answering the questions. Students first read the questions on their own and answered them without interruptions except that they were prompted to think aloud if they were quiet for a long time. After students had finished answering a question to the best of their ability, we asked them to further clarify issues that they had not clearly addressed earlier while thinking aloud.
Students' reasoning on questions in interviews were used as a guide to generate categories of difficulties and student responses on open-ended questions were coded into categories of difficulties. A subset of student responses on the open-ended questions were coded to determine categories of difficulties by two of the researchers separately. After comparing codes, any disagreements were discussed until full agreement was reached.

III. FINDINGS
Difficulty identifying the correct expression for expectation value: Table II shows that in response to question Q1, only 45% of the students selected the correct answer option D and correctly recognized that the identity operator is ∑ | ⟩ ⟨ | = ̂ and the expectation value of is ⟨Ψ|̂|Ψ⟩ = ∑ |⟨ |Ψ⟩| 2 . Table II also shows that 57% of the students selected options B or D that included statement 2. Interviews shed light on why 31% of the students selected the incorrect statement (3), ⟨Ψ|̂|Ψ⟩ = ∑ ⟨ |Ψ⟩ n , and will be discussed later in this section. For question Q2, a response was considered correct if the student inserted the identity operator, used an expansion of the generic state |Ψ〉 = ∑ | 〉 where = ⟨ |Ψ⟩, or conceptually reasoned that the expectation value is the sum of all eigenvalues of ̂ multiplied by the probability of obtaining that eigenvalue to obtain the correct final answer. Table II shows that only 29% of the UG students and 52% of the G students were able to obtain a correct expression for the expectation value of for state |Ψ〉 using a basis of eigenstates | 〉 and eigenvalues .
Question Q3 was graded using the same rubric as for the discrete case in question Q2. Table II shows that only 22% of the UG students and 53% of the G students were correctly able to obtain an expression for expectation value ̂ for state |Ψ〉 using a basis of eigenstates | 〉 and eigenvalues .
Below, we summarize the common conceptual and procedural difficulties involving the expectation value that were observed in written responses and interviews: Failing to reason about the expectation value conceptually: In interviews, students were asked to determine the expectation value and describe conceptually what the expectation value means. Very few students reasoned conceptually that the expectation value is the average of a large number of measurements on identically prepared systems to determine that ⟨Ψ|̂|Ψ⟩ = ∑ |⟨ |Ψ⟩| 2 . Most students used a formal approach to evaluate the expectation value. While some students followed correct procedures such as inserting the identity operator in terms of the eigenstates of the operator or expanding the generic state |Ψ〉 as a linear superposition of the eigenstates of the operator, many students who tried to use these methods got lost along the way. The fact that so few students were able to reason conceptually about how to determine the expectation value points to the fact that even upper-level UG and G students often prefer "plug and chug" methods as opposed to developing a coherent conceptual understanding that can facilitate the use of the simpler conceptual approaches (which are significantly less prone to error). G students were more facile in using the identity Table II. Percentages of students answering questions related to the expectation value. Percentages of students providing the correct answers are bolded. Q1 A (12%), B (12%), C (5%), D (45%), E (26%) Q2 29% UG students, 52% G student Q3 22% UG students, 53% G students operator to determine the expectation value than UG students. Therefore, the percentages of UG and G students answering Q2 and Q3 correctly in Table II is very different. However, written responses and interviews with UG and G students suggest that many of them did not realize that the expectation value is the average of a large number of measurements on identically prepared systems and they could have reduced their chances of making a procedural mistake if they had used a conceptual approach to find the expectation value.
Also, some students inappropriately interchanged the states |Ψ〉 and | 〉 when finding the expectation value. Interviews and written responses suggest that instead of recalling that a generic state |Ψ〉 can be written as a linear superposition of a complete set of eigenstates of an operator, some students thought that |Ψ〉 can be written as an eigenstate of ̂ when finding the expectation value of . For example, some students correctly wrote ⟨Ψ|̂|Ψ⟩ and then arbitrarily replaced the generic state |Ψ〉 with the eigenstate | 〉. One student who stated that |Ψ〉 = | 〉 in this context wrote: In addition to not understanding how a generic state differs from an eigenstate of ̂ and replacing |Ψ〉 with | 〉, this student (and many others) made several procedural mistakes. For example, without justification, the student introduced a summation over index i going from 1 to but the index i is never used in the expression he was summing over (he summed over but used the index when writing the eigenstate | 〉). Moreover, instead of using a Kronecker delta, he used a Dirac delta function, which diverges when ′ = . Another student, who arbitrarily replaced the generic state |Ψ〉 with | 〉, wrote 〈̂〉 = ⟨Ψ|̂|Ψ⟩ = ∑ ⟨ | ⟩ ∞ =1 . This student also introduced a sum (although it is over the index ) but did not justify where it came from. Interviews suggest that at least some students who incorrectly replaced |Ψ〉 with | 〉 introduced a summation because they remembered that the expectation value of an observable involves a summation.
Writing an incorrect expression for the expansion of | 〉 using a complete set of eigenstates of the operator ̂. Some students wrote incorrect expansions of |Ψ〉, e.g., |Ψ〉 = ∑ | 〉 or |Ψ〉 = ∑ | ⟩. Table III should be consulted for the specific percentages of students displaying this difficulty. For example, one student stated: "|Ψ⟩ can be expanded as a sum of eigenstates of ̂, |Ψ⟩ = ∑ | ⟩." This student incorrectly claimed that the eigenvalues of the operator ̂ were the expansion coefficients when |Ψ⟩ is expanded in terms of a complete set of eigenstates | 〉. If this were the case, the expansion coefficients will always be the same regardless of what |Ψ⟩ actually is. Another student reasoned that |Ψ〉 = ∑| 〉 and wrote: ⟨Ψ|̂|Ψ⟩ = ∑⟨Ψ|̂| ⟩ = ∑⟨Ψ| | ⟩ = ∑ ⟨Ψ| ⟩. Interestingly, this student did not change the generic state ⟨Ψ| in the "bra" state.
We note that this type of reasoning may have led some students to incorrectly select statement 3 in question Q1 in Table I. These types of difficulties demonstrate that students have some correct knowledge, for example, they know that one can write |Ψ〉 as a superposition of the eigenstates of a generic operator ̂ and use this linear superposition to find the expectation value ⟨Ψ|̂|Ψ⟩. However, interviews suggest that they often struggle to determine the appropriate expansion of |Ψ〉 or the coefficients of the expansion partly because they do not have a conceptual understanding of what the expansion coefficients mean.
Writing an incorrect expression for the expectation value: Some students wrote an incorrect expression for the expectation value, e.g., ⟨ | ̂| Ψ⟩ in which the "bra" and "ket" states are not the same. Table III shows the specific percentages of students displaying this difficulty in the openended questions. For example, one student wrote: . These types of difficulties indicate that many students are not aware of the fact that the expectation value is found by "sandwiching" the operator between the "bra" and "ket" states in which the expectation value is evaluated, i.e., ⟨Ψ|̂|Ψ⟩.
Attempting to use the identity operator but getting lost along the way: Some students were aware of the fact that one could find the expectation value by inserting the identity operator in the expression for the expectation value, but they had difficulty with the procedure and/or got lost along the way. Table III should be consulted for the specific percentages of students displaying these difficulties. For example, one common difficulty was an inability to distinguish between identity and projection operators. Other students had the correct expression for the identity operator but were unable to determine the expectation value correctly. For example, one student wrote the following: ̂= ∑ | ⟩ ⟨ |, ⟨Ψ|̂|Ψ⟩ = ∑⟨Ψ|̂| ⟩⟨ |Ψ⟩ = ∑⟨Ψ|̂| ⟩Ψ( ).
This student was able to correctly insert the identity operator but did not define Ψ( ) and left his final answer in terms of the operator ̂. Another student wrote: ̂= ∑ | ⟩ ⟨ |, ̂= ∫| ⟩⟨ | and ⟨Ψ|̂|Ψ⟩ = ⟨Ψ|̂∫| ⟩⟨ | |Ψ⟩ = ⟨Ψ| ∫̂| ⟩⟨ |Ψ⟩ = ⟨Ψ| ∫ ⟨ |Ψ⟩ . This student was able to correctly insert the identity operator, but then stated that ̂| ⟩ = without the state | ⟩ on the right hand side. Similar difficulties have been observed in other contexts as well. For example, prior research shows that some students believe that the Hamiltonian operator ̂ acting on an energy eigenstate | ⟩ yields the corresponding eigenvalue , i.e., ̂| ⟩ = [4]. Other students believe that the position operator ̂ acting on a position eigenstate | ′⟩ yields eigenvalue ′, i.e., ̂| ′⟩ = ′. Some students physically justify their incorrect responses of this type by claiming that the Hamiltonian operator acting on its eigenstate corresponds to the measurement of energy and should yield energy on the right hand side of the equation or that the position operator acting on its eigenstate corresponds to the measurement of position and should yield position on the right hand side.

IV. SUMMARY
Upper-level UG and G students had many types of common difficulties with the expectation value of an operator ̂ in terms of the eigenvalues and eigenstates of ̂ (e.g., when given that the states {| ⟩, = 1,2,3 … ∞} are the eigenstates of an operator ̂ corresponding to a physical observable with discrete eigenvalues ). Few students were able to reason conceptually about the expectation value and students often had many procedural difficulties in determining the expectation value of an observable in terms of the complete set of orthonormal eigenstates and eigenvalues of the corresponding hermitian operator.
Students must be given opportunities to build a good conceptual understanding of the expectation values of observables as well as develop the ability to calculate the expectation values. Instructors and researchers can use the student difficulties with the expectation values found in this study as a guide in developing curricula and pedagogies to help advanced students in QM courses develop a robust understanding of the expectation value.