The use of ACER to develop and analyze student responses to expectation value problems

In this study we use the ACER framework to investigate how students perform the expectation value in quantum mechanics. Students were given an exam question that required the use of the generalized uncertainty principle to find the lower bound of ∆Â∆B̂ for different angular momentum operators. This question requires students to use several mathematical tools, but in this study we focus on the expectation value. The data are analyzed using the ACER framework and we give special attention to students’ choice of mathematical representation. Of the four components in the ACER framework (activation, construction, execution, and reflection), we find that in our question the activation step presents the largest stumbling block for students. We also discuss the limitations of the ACER framework for this type of investigation.


I. INTRODUCTION
Quantum mechanics poses different challenges than other upper-division courses in that many of the concepts do not fit within the laws of classical physics and are often counterintuitive.Students must also learn a new mathematical formalism and become comfortable moving between different representations and notations.Much of the research on student learning of quantum mechanics has focused on student difficulties of topics such as energy measurements, time dependence, and the uncertainty principle (see Ref. [1] for an overview).Relatively little attention has been given to student mathematical abilities within quantum mechanics.
The ways that students synthesize their math knowledge while solving physics problems has become a common interest among researchers.The use of math in physics is vastly different than its use in mathematics [2].ACER was designed as an analytic framework to describe how students use various mathematical tools in advanced physics contexts [3].In this study we use ACER to both design and analyze mathematically rich questions in the context of quantum mechanics.
Quantum mechanics is a rich environment for studying student use of math in physics as multiple mathematical representations are commonly used.For instance, quantum states can be written in Dirac notation, matrix notation, or as position-space wave functions (among others).Depending on the representation, many problems can be solved using multiple methods.For example, it is possible to solve for the expectation value using matrix equations, Dirac notation, or integral expressions.In different contexts, one method may be preferred, but research shows that students tend to be more comfortable using a single notation and use it in most contexts [4].Our study is designed to probe how students solve for expectation values of angular momentum operators and which representation they choose to use.Solving the expectation value of an operator is a very mathematical exercise; therefore, we chose to use the ACER framework to both guide the development of the question we asked students, and in the analysis of the responses.This work provides insight into student reasoning when solving for the expectation value, as well as the suitability of ACER as a framework for our research.

II. METHODS
In this section we describe the ACER framework [3] and how it was used in this research.We also discuss the context of our study, including an overview of the two questions students were asked to solve.

A. ACER
ACER is a framework that can be used to help analyze how students use mathematical tools when solving physics problems.Wilcox et.al. used it to better understand how students used the Taylor series in upper-division classical mechanics courses, as well as direct integration and Dirac delta functions in upper-division electricity and magnetism courses [3,5].ACER organizes students' mathematical problem solving into four different categories: activation, construction, execution, and reflection.In this study, we not only use ACER to analyze student responses, but also to inform the development of our questions.
Within the ACER framework, an element refers to a small piece of a solution.Generally a question is operationalized for use with ACER by having expert solutions divided into elements that fit into the four categories.We have used ACER not only to analyze our data, but to guide the development of our questions.We began by identifying elements we wanted to see in students' solutions, and then created a question that included them.Once the question was written, we operationalized ACER to create a full list of elements in the solution.This list was revised after a first round of analysis of student responses to include elements missing initially.Our data analysis included coding for the prevalence of each element in the four categories.
ACER was designed to analyze questions that involve a single, well-defined mathematical tool, such as a type of integral or the delta function [3].Our study deviates from ACER in two ways: our question required students to use multiple mathematical tools (the commutator, the expectation value, and the absolute value) and there was more than one method that could be used to solve the problem.Reflect on your answer and, in a sentence or two, comment on why it does (or does not) make sense.
FIG. 1. Two problems based on the generalized uncertainty principle that were given to students in upper-division quantum mechanics courses.Question S was given to students at CSUF and CU Boulder on a midterm examination.Question L was given to students at CSUF on an in-class quiz the following semester.

B. Context for Research
Over the course of two semesters, we asked students enrolled in upper-division quantum mechanics to solve two questions based on the generalized uncertainty principle.The data were collected at two institutions: California State University, Fullerton (CSUF) and the University of Colorado at Boulder (CU Boulder).The instructors for both courses taught the first semester of upper-division quantum mechanics using the spins-first method.The same textbook [6] was used for both classes, and both courses used similar interactive materials (clicker questions and tutorials).Data were also collected from the second semester quantum mechanics course at CSUF that was taught by the same instructor using the same textbook.The main difference between these two institutions is the student populations; CU Boulder is a research intensive, selective institution and CSUF is a teachingfocused, masters-granting, Hispanic-serving institution.
We asked two different, although similar, questions on exams at CSUF and CU Boulder.Both questions are illustrated in Figure 1.
Question S was given on examinations at both CSUF (N F = 22) and CU Boulder (N B = 41) and is in the context of spin angular momentum operators ( Ŝx , Ŝy , and Ŝz ).
Question L was given on an in-class quiz at CSUF (N F = 18) during the second semester quantum mechanics course.It is very similar to Question S, except it is in the context of orbital angular momentum operators ( Lx , Ly , and Lz ).An additional difference between the two questions is that the state given in Question L was written in two different notations: position-space wave functions and Dirac notation.This allows us to see which notation students chose to use when solving for the expectation value.We also added a second part to the question in which students were asked to reflect on their answer.Note that Question L asks for the lower bound of ∆ Lx ∆ Ly in a context where the angular momentum is constrained to be in Lz .We did not see any evidence that this affected student responses.

III. RESULTS
While the questions we asked required students to solve for multiple mathematical tools, in this paper we focus solely on the expectation value.We analyze components of student responses for the four categories in the ACER framework: activation, construction, execution, and reflection.We report trends that were found in each category.Note that not all students are represented in the data set for each section.Responses for each category were only considered if the student attempted the relevant part of the question.For example, if a student did not attempt to execute the mathematics, they were not included in the execution data set.

A. Activation of the Tool
Activation of the tool involves recognizing what mathematical tool(s) are required in order to solve the problem.Our questions ask students for the lower bound of the product of two uncertainties.We intended to focus on the activation of the expectation value tool; however, in order for students to arrive at the expectation value component of the question, they must choose to evaluate the right hand side of the generalized uncertainty relation (∆ Â∆ B ≥ 1  2 | [ Â, B] |).When analyzing the data we quickly discovered that it was not clear to all students which side of the expression to evaluate.In this section we focus on the activation of an additional mathematical tool, the lower bound, since it is a necessary step in order to get to the expectation value component of the problem.
Question S: In this question, activation of the tool proved difficult for students, with only 72% (16 of 22) of students at CSUF and 83% (34 of 41) of students at CU Boulder attempting to solve the right hand side of the relation.Other students either solved the left hand side of the relation (three at CSUF and six at CU Boulder) or left the question blank.Additionally, there were several students (five at CSUF and one at CU Boulder) that started solving the right hand side, but before reaching the expectation value component they switched to solving the left hand side.Note that in this question, the two sides of the generalized uncertainty expression are equal, but this is not always the case.
Question L: Similar issues with activation were found in response to Question L, with 83% (15 of 18) of CSUF students correctly identifying that the right hand side needed to be solved.Most of the students that responded to this question also answered Question S in the previous semester, and this may be the reason more CSUF students activated correctly.The remaining three students either didn't attempt the problem or attempted to solve the left hand side.

B. Construction of the Model
For our questions, construction of the model involves choosing a representation and a method to solve the problem.We look at the representations chosen through the lens of structural aspects of quantum notations [4].In our questions, the representations and the methods used to solve for the expectation value are linked.The methods for solving for the expectation value are: the summation method ( Â = p i λ i , where p i are the probabilities corresponding to the eigenvalues λ i of the operator Â); direct integration ( ψ * (x) Âψ(x)dx); Dirac notation (from the equation Â = ψ| Â|ψ , with the use of eigenequation and orthogonality conditions); and matrix multiplication.In our analysis, we look for correct construction of the expectation value, as well as which method was used.We only consider responses from students that attempted to solve for the expectation value; that is, we only consider students that activated correctly (as described in the previous section).Students were considered to have correct construction if they set up the solution correctly, meaning they expressed the states and equations accurately in the representation of their choice.
Question S: In response to this question, students used one of two methods to solve for the expectation value: the summation method and matrix multiplication.The summation method can be seen as simplest method for this problem because the probabilities and eigenvalues were readily available, since the state was written in the Ŝz eigenbasis.Despite this, only three students at CU Boulder (and none at CSUF) used the summation method, as seen in Table I.Eight students at CSUF and two students at CU Boulder did not use either method as they either stopped solving the problem, made a mistake, or switched to solving for the left hand side of the equation before choosing a representation.The most common error in construction was a missing constant.
Question L: Students used three different methods to solve for the expectation value in this problem: the summation method, Dirac notation, and direct integration in the position representation.Because we provided the quantum state in both position representation and Dirac notation, students needed to decide which representation to use.This contributed to their choice of method when solving for the expectation value.Table II II.The number of students that used each method to solve for the expectation value on Question L. No method refers to students who stopped solving the question or made a mistake before they were required to choose a method.each method.All three of the methods were used by students in relatively equal numbers.Of the five students that did not select a method, two students claimed Lx = constant, without explanation.A further three students made an error or stopped working through the problem before choosing a representation.
The most common method to solve for the expectation value was different in the two questions.While most students used matrix multiplication in response to Question S, this method is not practical for Question L. We found that equal numbers of students worked in the position and Dirac representations in Question L. Note that the summation method did not require students to choose between these representations.

C. Execution of the Mathematics
Execution of the mathematics involves performing the mathematical operations necessary to obtain a result.In this section we again only look at the expectation value part of this problem.Depending on the choice of representation, solving for the expectation value can involve multiplying matrices, using orthonormality conditions in Dirac notation, or evaluating integrals.In this section we only consider students that have activated the tool and attempted to construct the problem.
Question S: Out of the eight students at CSUF that attempted to construct the problem, only three students executed the problem correctly.At CU Boulder, most students executed the problem correctly (28 of 32).The students that executed the problem incorrectly were missing a constant factor (four at CSUF, one at CU Boulder) or had incorrect algebra (one at CSUF, three at CU Boulder).
Question L: Out of the ten students that attempted construction, five had the correct execution of the mathematics.The representation that had the most errors in execution was position space.Two of these students made errors involving complex exponentials and two students stopped solving the question or made an error before completing the computation.When using Dirac notation one student left off the complex conjugate of the wave function, solving for Lz |ψ instead of ψ| Lz |ψ .

D. Reflection on the Result
Reflection on the result involves making sense of the answer physically, as well as checking for errors.Reflection is an important aspect of problem solving and tends to be present in experts' solutions and absent from students' solutions [3].While Question S did not explicitly ask students to reflect on their answer, Question L was designed to examine how students reflect on their answer when prompted.
Question S: Even though the students were not prompted to reflect on their answer, one student at CU Boulder solved the problem using two methods.While not a formal reflection, this students took steps to check their final answer.
Question L: This question prompted students to reflect on their answer.Even though some students did not finish the problem, 16 of 18 students answered the reflection portion of the question.The most popular response was that a non-zero answer made sense due to the uncertainty principle of noncommuting operators.The second most common reflection was that the answer needed to be real.
Even though most of the students responded to the reflection part of Question L, there was no evidence that students used this opportunity to meaningfully reflect on whether or not their answer made sense.Rather, the students seemed to use the reflection as an opportunity to justify their answer.

E. Limitations of ACER
The ACER framework focuses analysis on a single mathematical tool within a physics problem.However, in most problems, there are multiple mathematical operations that must be performed.In our question, for example, students must determine a commutation relation and take the absolute value of the result in addition to the two other mathematical tools discussed earlier (the lower bound and the expectation value).In order to get a complete picture of student responses to our question, we need to expand our analysis to include the other mathematical tools.We have included some patterns in student responses below.
Absolute values: The final step to both questions is to take the absolute value of the resulting complex number.We did not expect students to have trouble with this; however, five students at Fullerton and four at CU Boulder left an imaginary i in their final answer on Question S. No students made this error in Question L.
The commutator: Most students wrote the result of the commutator either from their equation sheet or from memory (28 of 50 for Question S and 9 of 15 for Question L).Most of the students who solved the commutator did so correctly (22 of 50 for Question S and 2 of 15 for Question L); however, we did not observe any patterns to the mistakes that students made.

IV. DISCUSSION
In this study we used ACER in a novel way to purposefully create questions that would highlight all four aspects of the ACER framework.We found the activation and reflection categories most challenging.In an attempt to write a contextrich question, we introduced an additional mathematical tool to the question: the lower bound.We did not expect students to have trouble with this concept as we were not aware of any physics education research literature on this topic.However, we found it challenging for students and it was the main reason students failed to activate correctly.
There were multiple methods that students could use to solve for the expectation value.For Question S, almost all students chose to work in the matrix representation, even though it required more calculations than the summation method.It is possible that students looked at the formula for the expectation value and went straight to the matrix method without recognizing that there was an alternate, maybe simpler, method.In contrast, for Question L, students were equally split between the three main methods.
We know that students rarely spontaneously reflect on their answers [3].We had hoped that an explicit prompt in the question would help.However, there was little evidence that students were meaningfully reflecting by using this as an opportunity to determine whether their solution was correct or made physical sense.Rather, it appeared that most reflections were simply a factual statement about their answer.
Future work will include performing student interviews as well as analyzing the data using methods other than ACER, in order to get a more in-depth look at student thinking, especially during activation and reflection.

Question S : 1 2= 1 p 1 2
Using the generalized uncertainty principle, Â B |h[ Â, B]i|, find the lower bound on Ŝx Ŝy for | i = 1 2 e i⇡ |+i + Consider a particle along a ring (with V = 0) with the state: | i is the lowest possible value of Lx Ly for this state?(Recall that Â B |h[ Â, B]i|.) b.

TABLE I
shows the number of students that chose . The number of students, broken apart by institution, who used each method to solve Question S. Note that students could have used more than one method.No method refers to students who stopped solving the question or made a mistake before they were required to choose a method.