Tracking the referent system to understand students’ math modeling processes

Math modeling is an iterative process where students use their observations and reasoning abilities to create and refine predictions or explanations about a single referent system (e.g., a lab apparatus). However, modeling may be complicated by the presence of multiple problem representations in an activity (e.g., both a word problem and a video demo), in which case students have multiple referents to consider. To study students’ modeling behavior in these situations, we conducted think-aloud interviews with pairs of students around a kinematics activity. While creating process maps of students’ modeling, we found that tracking students’ “system focus” was critical to interpreting their reasoning and decisions. The model referent is no longer seen as a static object; instead, it is a dynamic object that changes in response to both structured prompts and spontaneous events. This concept gives rise to practical insights into activity design and implementation.


I. INTRODUCTION
An important component of problem solving in the classroom and workplace is mathematical modeling [1]. Mathematical modeling is a process where a physical system is translated into mathematics and the end result allows one to assess the impact of specific parameters on the outcome of that system [1][2][3][4].
Most approaches to introducing math modeling within instruction involve comparing and contrasting multiple models as part of the modeling process. For example, one modeling framework that was developed for physics laboratory work centers on establishing agreement between observations and mathematical predictions [5]. To accomplish this, students make comparisons between the two, and if the observations do not match the predictions, they can revise the apparatus itself, the equipment to measure the apparatus, the math model describing the apparatus, or the math model describing the measurement equipment. Students engage in this cyclical revision process until their refined observations match their refined predictions, comparing different models of the system along the way.
Hestenes also provides a theory of the modeling process in the physics classroom [4,6]. In his work, he describes how multiple models can exist for one referent system: "every model is only a partial representation of an object, sometimes ignoring obvious properties." In other words, one referent can have multiple models, simply because each model does not describe every aspect of the system at hand.
One thing both descriptions of modeling have in common is that both focus on situations where students have a clear and well-defined system to model. The Modeling Framework for experimental physics was designed to understand the modeling process a student would carry out with an apparatus in front of them, and Hestenes' work also focuses on modeling objects in the real world. In both theories, there is little question of what should be the referent of students' models.
In practice, there are many physics activities that explicitly or implicitly contain several referents (e.g., word prompt, simulation, video demo, physical demo) that students could choose to model. Consider an activity where students are given a written description of a cannon shooting a projectile off a cliff, followed by a demonstration from a professor where a ball is fired from a desktop by a small projectile launcher. While these are certainly analogous referent systems, they are not identical, and students may have to compare and contrast these systems while modeling.
We hypothesize that student math modeling is complicated by the presence of multiple referents in an activity. Rather than simply having to compare between several different models of a single referent, students have several referents to consider along with the different models of each of those referents. This idea is depicted in Figure 1. We give the name "system focus" to the idea that students shift their focus between multiple referent systems and their associated models One referent Two referents Figure 1. A contrast between modeling with a single referent and modeling with two analogous referents (e.g., word prompt and a demo apparatus). Each arrow indicates a potential comparison that students could make. Models A and B correspond to two models of the same referent (e.g., an original and revised model). Having multiple referents significantly increases the number of comparisons available to students and complicates the modeling process.
throughout an activity. By studying students' in-the-moment modeling process through the lens of system focus, we hope to better explain and understand how students go about modeling.

A. The activity and data collection
To study students' in-the-moment modeling process, we designed and implemented a multi-phase physics activity. The structure of the activity along with the the main systems present in each section are presented in Table I.
The activity was given to pairs of students. To record the interviews, a camera on a tripod was placed such that both students were in view. However, this camera was not able to see what students were writing. To that end, two liverecording pens were used to record students' writing in a video format, allowing us to sync the camcorder's footage with what students were writing at any given moment.
Additionally, participants completed an online survey several days before the interview, which asked about demographic information along with several physics questions to gauge students' knowledge of kinematics and physics. Seven interviews were completed, but the data presented in this study is drawn from the final three, which represented the final refined version of the activity. Group A consisted of Student A1, a second-year male electrical engineering major, and Student A2, a second-year male computer engineering major. Group B consisted of Students B1, a male fourth-year Segment of Activity

Dominant system focus
Students are given the following prompt: a zookeeper is trying to hit a monkey with a dart. Knowing that the monkey will drop as soon as the dart is fired, where should the zookeeper aim? See Figure 2.

Original prompt
Students are shown a video demonstration where a physics teacher raises a stuffed monkey, points a cannon directly at it, and fires a projectile at it from roughly forty feet away [7].

Video demonstration
Students are asked to explain the video using mathematics.
Video Students are given an apparatus (See Figure 3), and told to re-create the video.
Apparatus Students are asked to revise the original math model based on their experience with the apparatus. Revised prompt, apparatus Table I. Structure of the activity. Students progress through multiple representations of kinematics problem, eventually use their experience with an apparatus to revise their original model used to describe the system. Figure 2. The picture associated with the initial prompt given to students. The students are asked whether the dart should be aimed more like (1), (2), or (3). Figure 3. The materials given to students to replicate the experiment from the video: a projectile launcher (Pasco part ME-6800), an eraser, and ring stands. physics major, and B2, a fourth-year physics major who did not disclose their gender. Group C consisted of Student C1, a male third-year physics major, and C2, a female third-year physics major. Information on students' race or ethnicity was not collected.

B. Analysis
We used the theory of epistemic games to think about and describe student behavior. An epistemic game is a strategy for problem solving and analysis that follows a specific epistemic form, first introduced by Collins and Ferguson [8]. Epistemic games have starting and ending conditions along with a list of moves that are played during the game. As such, we kept track of the various goals that students had along with the actions students took to achieve those goals. As goals, representations, and moves were noted, one researcher (the first author of this paper) described the data through an open coding process, such that every segment of the interview was assigned a phrase that encapsulated the events of that segment [9].
After the researcher had created the open codes, they created a map for each interview that laid out the actions taken by students throughout the model revision portion of the activity, which is the final segment shown in Table I. These maps tracked students' goals, moves, and epistemic forms throughout the activity in a visual fashion.
The maps were then discussed and reviewed with other researchers. While analyzing these modeling maps, we noted that students frequently changed which referent system (e.g., word prompt, apparatus, demo video) they were focusing on, and this corresponded to changes in reasoning patterns. This prompted another review of the data, where the interviews were coded again by the same researcher, this time with a focus on the system that students were thinking about as they worked. Particular attention was placed on cases where the students' focus shifted and where their focus was important to understanding their actions. These cases were then again compared and reviewed with the research team to understand how system focus influenced students' modeling processes.

III. RESULTS
We present several results showing how system focus and students' reasoning are interrelated in this activity with multiple referents. First, we note that system focus is often guided by the structure of the activity, and that acknowledging this relationship is useful from an activity design perspective. Second, we observe students must often reconcile and compare multiple referents, shaping their decisions and process. Finally, we make several specific claims about the nature of the physical apparatus as a referent and how an apparatus shapes student reasoning.
A. Tracking structural changes in system focus Keeping track of system focus is crucial to understanding students' actions while modeling. In our activity, the students have four possible system foci: the hypothetical system laid out in the original prompt, the video demonstration, the physical apparatus, and the revised hypothetical zookeeper/monkey system that students create as part of the model revision process. We observed examples of all four system foci in each interview.
To see the value in system focus, consider the following quote from students during the activity: B1: But you need [to give] the person more time to react, so you wanna aim higher, because if the person is reacting to the sound here they're more likely to be slower. Like you're right that if you aim lower it will hit the thing lower.
B2: So they won't fall as far, so that thing will have to be higher. Yeah, no, you're right.
Without more context, it's nearly impossible to tell whether the students are considering the apparatus they worked with or the revised, hypothetical zookeeper and monkey. These referent systems, while having many analogous features, have key differences. The zookeeper and monkey system is purely hypothetical, whereas the apparatus is physical and concrete. The apparatus therefore has a behavior that is controlled and observed by the user and constrained by the design of the apparatus. The hypothetical system's behavior changes depending on the assumptions and physical principles applied by the students. To fully understand what the students are doing, it is necessary to consider the system they are focusing on.
The structure of the activity plays a significant role in determining students' system focus. Each part of the activity naturally leads students to consider different systems, as outlined in Table I. Thus, shifts occur in between each of the major segments of the activity, which correspond to the rows of Table I. For example, the activity (with supplemental prompting from the interviewer) directly asks students to use their experience with the apparatus to revise a mathematical model:

Interviewer: So what do you think? How was this [the apparatus] different [from the mathematics]?
A2: Timing, the fact that this is not an instantaneous mechanism. This takes time for the spring to decompress and force the ball and accelerate it. So that's something that you have to take into account...you could have like a time delay for from the time that they start shooting to the time that it actually gets launched.
In this example the student considers two referents (the apparatus and the original zookeeper/monkey hypothetical scenario) and one model (the original math model describing the scenario). This quote exemplifies the types of complex comparisons modeled in Figure 1, where students must not only work with multiple models, but multiple referents. The next section describes these comparisons in more detail.

B. Reconciling and comparing multiple referents
We find that students in an activity with multiple referents must reconcile the information they receive from those referents, potentially treating one referent as "dominant" or more valid.
For example, the following is an excerpt from group C. While working with the apparatus (Fig. 3), the students tried several times to launch the ball and drop the eraser at the same moment, and then tried several times to drop the eraser slightly after the ball was launched. They observed the ball going above the eraser in some trials but below it in others. C2: Well I think, I just think we were going about doing it not the right way those times in general. Like, we were, it was, I think there was just a lot more uncertainty in how we were doing the experiment in the first way [without an intentional delay]. And so doing it just on the sound [intentionally including a delay] is much closer than the real world thing would be, for the monkey.
This example shows the students comparing between two referents: the apparatus and a revised version of the hypothetical zookeeper and monkey system. The students must reconcile and synthesize the predictions of their revised version and their observations. Student C2 argues that the revised system should supersede some of their observations, because their observations were based on a system that is not exactly analogous to the hypothetical situation. The concept of system focus helps to explain this decision: Student C2 is chiefly focused on the behavior of the hypothetical revised system, and considers the apparatus to be a means for understanding the revised system. This is an example of the referent-referent comparison depicted in the lower half of Figure 1.
In the presence of multiple referents, students also may discuss how to go about the modeling process, or decide what it means to go about modeling. The following example occurs after the interviewer asked the students to revise the model. Is that what you're talking about? 'Cause in that case we're not modeling the same system anymore...we assumed that both drop at t = 0. B1: ...one of the assumptions we're making here is that they're actually dropped at the same time. It's one of our conditions, but really it's an assumption that we're making that happened.
The students engage in a discussion of what it means to revise a model and explicitly mention that there are several referents to consider in the activity, each with different assumptions. B2 is confused by this, but B1 points out that changing an assumption is equivalent to changing the referent and the model that goes along with it. This exchange shows a certain metacognitive quality, where students reflect on and discuss the modeling process itself.

C. Supporting student reasoning with an apparatus as referent
Each referent in the activity possessed different qualities, so each referent shaped student reasoning in a different way. We found several examples of the apparatus uniquely shaping students' modeling process.
The apparatus in this activity serves to make students aware of features that they may not have considered previously, as shown in the following excerpt where students are setting up the apparatus: Interviewer: And [the launcher] has, uh, three velocity settings by the way.
B1: Uh, well we found that it was independent of velocity, right?

B1:
The equations didn't have any initial velocities in it.
In using the ball launcher the students are presented with a choice about the launch speed. This choice leads to a recognition that the initial velocity of the cannon is irrelevant in terms of the ball hitting the eraser. Despite having already derived this result, students did not recognize its significance until confronted with using the apparatus. We find similar discussions of the initial velocity in all three interviews.
Another way that the apparatus supports student reasoning is to introduce the importance of a time delay as a key feature in a revised model. The following quote occurs when students are trying to get the ball to hit the eraser: C1: ...we will try to do the best that we can. Alright, ready? 1, 2, 3, go. [ball misses eraser] Dammit. That was close though! It's so close! It's just our reaction times. Okay, so we can say that about the real-life ones.
Here, the students encounter difficulty in getting the apparatus to recreate the video. They then take this difficulty and apply it to the hypothetical zookeeper/monkey scenario, saying that there may be a time delay in the real-life situation. The apparatus as a referent served the valuable function of highlighting which revisions to the original math model are most salient in light of real-world considerations. Attention was focused on the time delay, while attention on other features, like air resistance, decreased.

IV. CONCLUSIONS
Through an in-depth analysis of a modeling activity, we discovered that a student's current system focus helps to explain why they make certain decisions. While there are a variety of referent systems that students could focus on in an activity, they do not consider all of them equally at all times, and so a student's focus shapes their actions in the moment. It may seem trivial that the questions asked by an activity are the biggest determining factor in students' system focus, but keeping this in mind can be beneficial when designing an activity. The results suggest that adding demonstrations or simulations into a lesson may increase the complexity of model-based reasoning, particularly the comparison and revision phases, because of the large number of possible comparisons (again, see Fig. 1.) This is exacerbated by the ambiguity as to which of the referents is deemed as "correct" or serves as the standard by which to judge the other referents.
System focus can be understood within the epistemic games framework because a students' system focus affects their goals, ending condition, available moves, and the epistemic forms or representations that are used. For this reason, when studying or characterizing epistemic games involving modeling with multiple referents, we recommend tracking students' system focus.
In the classroom, paying attention to what systems students are working with and making comparisons between could be invaluable during activities where demonstrations and word problems are simultaneously present. If students are stuck or confused, then an intervention by a TA or instructor will be more effective if they are aware what system or systems the students are focused on. In addition, when designing an activity, keeping system focus in mind can help ensure that the prompting questions actually target the desired referents and encourage an analysis of the most relevant comparisons.
Future work in this area could include a similar analysis of other activities that contain higher-level physics content or with different types of referent systems (e.g. computer simulations).

ACKNOWLEDGMENTS
This work was partially supported by NSF Award DGE-1561493.