Comparing undergraduate and graduate student reasoning on a conceptual entropy questionnaire

,


I. INTRODUCTION & BACKGROUND
The notoriety of entropy as one of the most esoteric physical quantities makes it an ideal focus for a study of undergraduate reasoning with thermal physics concepts. Furthermore, entropy has a wide range of relevancy across the fields of biology, chemistry, physics, and engineering, yet is not typically covered in the undergraduate physics sequence until late in the upper-division.
As opposed to the lower-division, relatively few studies have explored undergraduate reasoning with entropy at the upper-division level. One study by Leinonen et al. examined the consistency between undergraduates' use of the second law of thermodynamics at the microscopic and macroscopic levels, and found that their reasoning was largely consistent, though some students treated entropy as a conserved quantity [1]. A second study identified resources students used when thinking about entropy and thermal equilibrium. It also observed students treating entropy as a conserved quantity in addition to noting that many students struggled to connect entropy with multiplicity [2]. Our research complements these two prior studies which occurred at non-physics-PhDgranting institutions. Other studies have analyzed how undergraduates compared changes in entropy of isothermal and free expansions of an ideal gas [3], and student reasoning on tasks related to heat engines and the Carnot cycle [4].
With the exception of the study by Leinonen, the research at the upper division has focused largely on the macroscopic perspective of entropy. The macroscopic, or thermodynamic, perspective considers entropy as a property of a system that changes when heat flows into or out of the system. Complementary to the macroscopic view, the microscopic (or statistical) perspective of entropy relates entropy to the multiplicity of microstates (Ω) through Boltzmann's law: S = k B ln Ω. Our goals for this research are to expand the literature on student reasoning in thermal physics by focusing on the microscopic perspective of entropy and the connection between entropy and probability which has received little attention within PER [5], and to provide a pseudo-longitudinal per- spective on student reasoning through a comparison of undergraduate and graduate student reasoning. This research follows a study on graduate student reasoning on entropy and related concepts [6,7]. We presented undergraduates with the same set of questions previously posed to graduate students, and this article focuses on part of the interview involving a system of a neutrally buoyant string waving in a bath of water-the same section of the interview discussed in our prior article on graduate student reasoning [6]. In this system the microstates are directly visible, and conceptualizing the macrostates requires more abstraction. This contrasts with many typical physical systems (such as ideal gases and Einstein solids) used to discuss the connections between entropy, microstates, and macrostates. This article will summarize the undergraduates' responses to the questions about the string system and compare them to the responses of the graduate students.

II. METHODOLOGY
To investigate how students reason about and solve problems involving entropy, we conducted think-aloud interviews [8] with upper-division undergraduates on a question designed to probe students' understanding of macrostates, microstates, and entropy in a novel context so that students could not simply respond by recalling results from a familiar system. The question presented interviewees with 3 cartoons which it described as different "'conformations' (meaning 'positions' or 'arrangements')" of a thin string waving in a bath of water (see Fig. 1). A preamble to the question told students that the string and water had equal densities so the gravitational force was completely cancelled by the buoyancy force, and that net external forces could be ignored.
The images in Fig. 1 represent microstates of this system, so by the fundamental assumption of statistical mechanics, they are all equally probable conformations of the string. However, in part A, we expected that some students might rank the state in Fig. 1a as less likely than either of the states in Fig. 1b or Fig. 1c, due to an intuition that the more 'disordered' looking configurations would be more favorable, a confusion the problem was designed to examine and was observed in the graduate student interviews.
Part B probed students' understanding of macrostates, as well as their creative ability to define macrostates in a novel, and likely unfamiliar, situation. Unlike canonical situations such as coin flips and Einstein solids, many acceptable definitions could be applied to this system (such as the distance between the ends or the number of bends), and we were interested in seeing the range of student-generated ideas. Generally, we considered any macrostate classification that outlined a means of partitioning microstates as correct. Asking students if the figures depicted macrostates or microstates was intended to reveal whether students could articulate the difference between the two terms, which is important for understanding entropy from a statistical/microscopic perspective. By following up immediately with the prompt to re-rank the probabilities in part C, we could see whether students might change their answer from part A after explicitly prompted to think about microstates and macrostates.
Finally, in part D, we wanted to see whether students could consolidate their reasoning about microstates and macrostates as well as articulate that the multiplicity of a macrostatei.e. the number of microstates within that macrostate, Ωdetermines its entropy via Boltzmann's law.
In total, ten undergraduates were interviewed: eight interviews were conducted in the Fall 2020 semester, and the remaining two were conducted in the Spring of 2021. All interviews were conducted through Zoom with the questions posed to students through a shared Google Jamboard. An interviewer (NC) was present to answer questions from the interviewees, prompt the students to verbalize their thinking, and ask students to further explain their reasoning. Before beginning the interview questions, students were encouraged to ask clarifying questions, to verbalize their thinking, and to read the interview questions out loud.
The first two students interviewed in the Fall 2020 semester were conducted with non-CU students, one from a mediumsized research university (this was the only student who had fully completed a thermal physics course prior to the interview) and the other from a large R1 university. The remaining eight students (six from the Fall of 2020, and two from the Spring of 2021), were students currently enrolled in the thermal physics course at the University of Colorado Boulder. All relevant content had been covered and tested in the course by the time students participated in the interviews. In both semesters author BRW was the course instructor. All students are referred to by pseudonyms.
Interview transcripts were automatically generated by Zoom. The drafts were then manually checked and edited against the recorded video. Interview transcripts were coded with a combination of a priori codes generated from the analysis of the graduate student interviews and new emergent codes based on the undergraduates' responses to question prompts. The coding methodology was inspired by Hammer's resources framework [9] for identifying the conceptual reasoning elements students employed in thinking through this problem. Responses were summarized and analyzed to identify the range of responses among the undergraduates and the commonalities and differences between them and the graduates.
After initial coding by the first author, all three authors collaboratively reviewed sections of interviews to verify code assignments and reach agreement on difficult-to-code passages.

III. RESULTS & DISCUSSION
In this section, we first present students' rankings of the probabilities of the three string configurations (covering student responses to parts A and C of the strings task), then we explore the students' reasoning about microstates and possible macrostate definitions (covering parts A and B). Finally, we discuss the students' reasoning about the entropy of the string and the extent to which they connected the idea of entropy to the concept of multiplicity (part D).
In the graduate responses to the questions about the string system, we observed a preference of students to initially rank the probabilities of microstates unequally, but were largely able to correct their rankings upon further consideration of the question. The graduate students generated a robust collection of possible macrostate properties, with many students giving multiple ways of partitioning microstates into macrostates. Some graduates appeared to project the likelihood of a macrostate on to constituent microstates, and some appeared to conflate macrostates with macroscopic objects. Throughout the question few graduate students related entropy with disorder, and half did not conceptually connect entropy with multiplicity [6].

A. Probability Rankings
As summarized in Tab. I, a majority of students (N = 5) settled on the correct ranking on all states being equally likely. Furthermore, all five of these students (Emma, Frank, Greg, Henry, and Jessie) alluded to the fundamental assumption of statistical mechanics by correctly claiming that all microstates of a system are equally likely, though none of these students referred to the fundamental assumption by name. This proportion was roughly similar to the proportion of graduate students implicitly or explicitly utilizing this idea.
Initially, the second most preferred ranking was a version that put the state in Fig. 1a as the most likely state. Both Abby and Bill argued for this based on the idea that forces from the water on the left and right side of the string should cancel out. This symmetry argument somewhat resembles an association between equilibrium and 'order' found in prior research [2,10]. In the final rankings, Dewei and Bill joined Isaiah in choosing the intuitive ranking of C > B > A. In his reasoning for the switch, Bill appeared to think that the microstates associated with a more probable macrostate were themselves more probable: Bill: If I think these are all microstate pictures [...] then like, I don't think any microstate is necessarily-But then I guess it's more likely that it's in this macrostate, so then it's the microstates of it that are more likely.
This reasoning echos our prior observation that multiple graduate students appeared to project macrostate characteristics onto constituent microstates [6].
In the interviews with graduates, four out of the eight students initially ranked the probabilities of the three snapshots from Fig. 1 by the intuitive ranking. Though, after considering part B of the question, many graduate students migrated to the correct answer (in total, six of the graduates settled on the correct ranking), and none settled on the intuitive ranking [6]. Three out of the ten undergraduates changed their answer, with two settling on the intuitive ranking. None of the undergraduates switched from an incorrect to a correct ranking after considering part B of the prompt.  Fig. 1. The second column, labeled 'Micro/Macro,' indicates whether the student identified the states from Fig. 1 as microstates or macrostates, and the instances of 'macrostate' were italicized for visibility. Bold entries in the final column indicate rankings that changed after students considered part B of the question.

Student
Initial The majority of interviewed students identified the states in Fig. 1 as microstates. This was often accompanied by the correct reasoning that a microstate is a particular, fully specified configuration of a system, and that a macrostate is a grouping of configurations that are in some way the same. For most students, this understanding was sufficient to generate a macrostate property for the string system. Greg, however, articulated a largely correct, abstract description of the difference between macrostates and microstates, but could not think of a macrostate property for the string: Greg: A macrostate, is like, if all the configurations were the same somehow, then the multiplicity of that macrostate would be greater than one. So, if there's multiple ways of arranging it, but if the way that they're arranged are the same, then that's a macrostate. But the concept that I'm having trouble with is, this probably applies to gases, too, but like, how they can fit into a macrostate.
Dewei, the only student who mentioned multiple macrostate properties, seemed to be reasoning by making a direct analogy to an ideal gas since earlier in the question, Dewei had invoked the ideal gas law when reasoning about the probability ranking, and talked about particles when discussing the difference between microstates and macrostates: Dewei: A microstate is like a particle, or an energy of a particle. It's single part. And a macrostate is like a group of particles or the group of particles' behaviors.
Furthermore, this reasoning seems to conflate macrostates with macroscopic objects, a difficulty also observed in the graduate interviews. When discussing whether the cartoons were microstates or macrostates, Carson also appeared to associate macrostates with macroscopic objects: Carson: I would say they're macrostates only because they're something that the entire-If we define a system as the string, and [...] we're able to identify the entirety of the string and what it's experiencing-not just what is the end of the string experiencing, what is the middle of the string experiencing-instead we're just saying the entire string is this, is experiencing this force of tension.
Carson's macrostate definition, the tautness (or tension) in the string, was also briefly considered by one of the graduate students before thinking of a more appropriate definition. In reality, there is a different force of tension at each point along the string, so tension is a somewhat indefinite way to define macrostates. However, Carson's reasoning that a macrostate property involves some macroscopic aspect of the system is valid, though it seemed to take for granted that the tension in the string was uniform across the entire string, as is typically the case in introductory physics.
The seven remaining undergraduates (Abby, Bill, Emma, Frank, Henry, Isaiah, and Jessie) all identified an appropriate property that could define macrostates of the string (see Table I). Each of these seven students, plus Carson, only gave a single macrostate definition and did not attempt to generate any additional ways of classifying macrostates. On the other hand, the graduate students we interviewed often generated multiple macrostate definitions spontaneously. The number of turns/curves/bends in the string was the most common property generated by the undergraduates, with four students giving this definition. Variations on the distance between two points on the string, given by Bill and Frank, was the next most common and one student, Abby, came up with the net horizontal deviation of the string. In the graduate student interviews, the three macrostate definitions of number of turns, net deviation, and distance between points were each generated by four graduate students.

C. Reasoning about the Entropy of the String
In total, seven undergraduates either explicitly stated that entropy was related to the number of microstates or mentioned Boltzmann's law in part D. Several of these undergraduates, though, expressed some discomfort about the (seemingly) infinite space of possible conformations: Emma: So, like, I don't think this is correct, but, I almost feel that defining entropy doesn't have a lot of meaning, right, because if we were to define it as our usual k B ln Ω then kind of saying like, there's infinite entropy, but-I guess, like, that doesn't sit very well with talking about how we add entropy...
In addition to Emma, Frank and Henry shared this concern that there is infinite entropy when an infinite number of microstates correspond to a single macrostate. All three of these students were among the strongest performing students on the entire interview, so this difficulty is not linked to poor performance. None of the graduate students expressed this same concern regarding an infinite number of microstates resulting in infinite entropy. It is unclear if this is because the graduate students understood a resolution to issue of a seemingly infinite number of microstates (e.g., via a quantization argument) or did not realize there was a problem.
The other three students not mentioning Boltzmann's Law or multiplicity made associations between maximizing entropy and finding the system in "its most natural state," "most probable state," or the "chaos state" (Abby, Carson, and Dewei respectively).
Interestingly, Emma also worried about what the equilibrium state of this system would look like: Emma: There is no, like, preferred configuration then because there are no net forces and the string will probably just keep moving at random and probably never reach an equilibrium state.
The equilibrium state of a system is a macrostate. Thus, it is possible this difficulty arises from the fact that at any moment an observer sees the string in a microstate as a result of the extra step required in conceptualizing the macrostates. Two graduates appeared to have a related difficulty in thinking that the entropy of the string was constant and did not change as the string's conformation changed [6].
In the graduate student interviews, four out of the eight students made the connection and articulated the link between entropy and multiplicity (compared to to 7 of 10 undergrads). Other studies have seen upper-division undergraduates struggle to connect entropy with multiplicity [2,11], so the observation here that many undergraduates successfully made this connection stands out. In similarity with the graduates, however, besides a brief comment by Isaiah and Dewei's mention of the "chaos state," the undergraduates did not spontaneously associate entropy of the string with "disorder."

IV. CONCLUSIONS
This study presented undergraduates with a novel system involving a string waving in a bath of water. Interviewees were shown images of the string in three different conformations and were asked to rank them based on the probability of finding the string each configuration. Then students were asked to think of a property which could be used to classify a set of macrostates of the system, and whether the images of the three conformations were microstates or macrostates. Students were then given the opportunity to re-rank the probabilities of the images. Finally, students discussed what was meant by the entropy of the string and how it relates to the number of conformations. We discussed undergraduate student's responses and compared them to the graduate student responses documented in a previous article [6].
On the whole, the responses of the undergraduates and graduates were similar. The most common probability rankings given by both populations were the correct ranking, an intuitive ranking, and a ranking with state A (in Fig. 1a) as the most likely. Many graduates migrated from incorrect to correct rankings while none of the undergraduates did the same; however, this could be an artifact of more undergraduates in this sample initially giving the correct answer.
The range of macrostate classifications generated by both populations was also very similar with the number of turns in the string, a distance between two points, or the net deviation being the most common. The graduate students, however, often gave multiple ways of classifying macrostates, while the undergraduates generally only gave one. Furthermore, we observed students in both populations projecting macrostate probabilities on to constituent microstates, and possibly conflating the concept of a macrostate with a macroscopic object.
When reasoning about what was meant by the entropy of the string, and how it relates to the number of conformations, majorities of both undergraduate and graduate students articulated a connection between entropy and multiplicity/a number of microstates. A small number of students in both samples expressed some confusion related to the equilibrium state of the string system. Associations between entropy and disorder were also uncommon in both groups despite being common in other published work regarding undergraduate conceptions of entropy [2,3]. As a point of contrast between the two sets of interviews, several undergraduates expressed unease that the seemingly infinite number of configurations corresponding to macrostates would result in infinite entropy, while none of the graduate students articulated this concern. This work is limited by the small sample size and the selection effect created by the solicitation of participants. In particular, this study occurred at a large, R1 university with a strong undergraduate physics program. Therefore, these results and findings likely do not capture the full scope of undergraduate reasoning on entropy. These findings add to the literature exploring student reasoning about entropy, particularly from the microscopic perspective, and provide a unique comparison between graduate and undergraduate reasoning.