Question characteristics and students’ epistemic framing

Problem-solving is an important skill for undergraduate physics students. In addition to having the basic skills and background knowledge, students’ epistemic framing can play an important role in their ability to solve physics problems. As part of a bigger effort in understanding how to help instructors to better facilitate students’ work, we would like to find out when students proceed through problems in similar ways and the potential relationship between question characteristics and students’ framing to these questions. In this study, we use a two-dimensional theoretical framework (CAMP framework: Conceptual Physics, Algorithmic Physics, Conceptual Math, Algorithmic Math) to analyze and compare different students’ framing to the same set of questions of research-based tutorials in upper-division electricity and magnetism. We present a case study of comparing the self-reported framing of two students as they work through the same set of tutorial problems. Preliminary analysis suggests a correlation between question characteristics and student epistemic framing.


I. INTRODUCTION
Over the last few decades, research across disciplines has led to significant progress in identifying fundamental principles of learning as well as using these principles to guide effective teaching practices [1,2]. One such area of such research is investigating students' reasoning on using math in physics. Being able to convert a word problem into mathematical representations is essential in problem solving.
Previous research shows that students' epistemic framing can have a major impact on how they use mathematics in physics problem solving. Epistemic framing refers to how an individual perceives a task at hand and determines what knowledge and tools are useful for completing that task [3]. As physics students solve problems, their epistemic framing connects how they perceive a task with the knowledge and cognitive tools they use to solve it. A student framing a physics question as merely an algorithmic task tends to focus on the mechanical behavior of equation plugging-andchugging, without necessarily thinking about the conceptual underpinnings and physical interpretations [3,4]. Sometimes students' difficulties in upper level courses may be the result of unproductive framing rather than a fundamental inability to solve the problems or misconceptions about physics content [5].
Prior work on students' framing has focused on their frames in the moment of problem solving. Comparatively little work, however, has investigated how the wording of a question affect students' framing, particularly among written questions such as in textbooks and worksheets. Understand how the phrasing of a question affects students' epistemic framing could help answer the question of when do students solve problems in similar wayswhether it makes them more or less productive. Further, it can help instructors better facilitate students' problem-solving in class as well as developing curriculum materials that were framing informed.
As a first step towards investigating the links between student framing and written questions, we investigated a common research-based curriculum for upper-division electricity and magnetism (E&M) [6] to look for which kinds of questions elicit which frames and how. These tutorials are developed through an evidence-based and iterative process which considers common student ideas and responses to these questions; therefore, they represent a curated set of problems for which prior research suggests that students should solve these problems in predictable and similar ways. We analyzed student responses using a two-dimensional theoretical framework (CAMP framework) [7] that spans the algorithmic vs. conceptual ends and physical vs. mathematical ends. More detailed discussion on the CAMP framework is below.
Students' framing while solving problems is partially determined by the specifics of the problems they solve, so an investigation into how independent students frame as they solve problems has potential implications for how instructors notice students' frames and facilitate problem solving. In this preliminary work, we present a case study of comparing frames of two undergraduate students working through questions from the CU Boulder tutorials [6] for the upperdivision E&M. Similarities and differences of students' framing are analyzed to see the possible connections to question characteristics. Our two focal students are welltrained in noticing framing, and their reflective reports of their framing, coupled to their problem solving, form the data set we investigate in this paper.

II. THEORETICAL BACKGROUND
Many studies explored students' problem solving through an epistemic lens [3,4,8]. Epistemic framing is the process by which a student pares down the set of all their available knowledge, selecting (often subconsciously) a subset of their knowledge for the purpose of solving problems, constructing new knowledge, and evaluating what they know [9]. Bing and Redish introduced four epistemic frames to describe students' use of math in a physics context, namely physical mapping, calculation, invoking authority, math consistency [3].
Later work on student framing using math and physics has explored the differences between the conceptual physics, conceptual mathematics, and algorithmic mathematics frames [10]. By influencing the tools, knowledge and frequently the attitudes of the students, framing has a significant effect on student learning and problem solving [11]. Framing allows researchers to seek causal explanations of difficulties in students' problem solving. As instructors, we would like students to balance between algorithmic vs. conceptual frames and physical vs. mathematical frames; and to translate back and forth between different frames in order to solve a physics problem productively.
The model was then revised to expand two orthogonal axes as illustrated in Figure 1 below [7]. The horizontal axis describes students' activities towards the conceptual or algorithmic ends, while the vertical axis extends towards the physics and math ends. These orthogonal axes make up four quadrants, corresponding to four frames students may be engaged in problem-solving involving math. These four frames are described below. We use this two-dimensional theoretical framework (CAMP framework) in this study. Conceptual physics (CP) frame: Students are in this frame when discussing conceptual ideas of physics or properties of physics quantities/systems/scenarios. For example, students discuss the physics scenario at hand or make a plan for solving the problem. Students may also determine the direction of vector quantities, and discuss whether a quantity is constant or variable. Algorithmic physics (AP) frame: Students in this frame combine knowledge of the physical system at hand and appropriate mathematical formulation to set up the mathematical equations for solving the problem. For example, students may recall an equation or apply a general equation to a specific case or perform dimensional analysis to make sense of an equation or result. Algorithmic math (AM) frame: Students are in this frame when they perform algorithmic math procedures to obtain a result. For example, students may be computing an integral or solving algebraic equations. Conceptual math (CM) frame: Students are in this frame if they perceive the computation task at hand as an opportunity to apply mathematical rules and properties to avoid detailed and lengthy manipulation. For example, when students encounter ∫ , they reason that the result 0 is zero based on the symmetry of the sine and cosine functions instead of actually computing the integral.

Types of questions:
As mentioned previously, the questions studied are selected from CU Boulder tutorials which are research-based materials designed to help students develop key conceptual understanding and problem-solving in upper-division E&M [6]. We decided to choose these questions because of the wide adoption of these materials. The leading faculty on this project also use these tutorials in their own classes so there is a strong motivation to study these questions for practical reasons.
This project represents a multi-institutional study where graduate students and undergraduates from four institutions engaged in collaborative research around student framing in the tutorials. The project as a whole seeks to link students' problem solving moves in the tutorials to the instructor's interactions with the students. In this paper, we focus on how specific problems in the tutorials might affect students' framing using the self-reported framing from two undergraduate students who are part of the research team. These two students are from separate institutions and both completed their E&M coursework prior to starting research, they also received extensive training on framing before recording their own. Students first separately solved tutorial problems, then they self-recorded their own framing for each problem.
To prepare for this project, the students engaged in literature reviews and generative writing around framing. Generative writing is a process where students were asked to write generally and frequently to record their thoughts, synthesize papers, reveals holes in argument to fill with literature or analysis, and overall help to generate ideas. S1 read ten articles spanning over 155 pages of literature and wrote over twenty pages of literature reviews. S2 read six articles and wrote fourteen pages of literature reviews. Throughout summer 2019, both students attended the weekly intensive meetings with discussions of both the framework and how it might be extended to examine written materials.
Tutorial mapping process: -Rubric: Before frame mapping, the student team (including these two focal students) first trained on an existing rubric [11] for classifying classroom observations of students. The student team discussed the definition and added concrete examples for each frame. These two focal students calibrated their coding by comparing and discussing their coding with an expert who is a graduate student working on coding for one year during the weekly meetings.
In table I we list an example of the rubric students used. -Solving the questions and self-mapping: After the training stage, the students worked through five tutorials, and mapped their work using framing analysis [7,10], documenting their reasoning. A sample of students' solution and their notes of reasoning is provided below: Solution: Frame mapping notes: 3a. CM -to draw a coordinate system CP/AM -to drawing charges in a system that has two sin dependencies 3b. AP -Recall Qenc equation AM -Unpack the Integral on the right hand side 3c. CM -Exploiting sin behavior to quickly derive the answer to an integral 3d. CP -I believe that the overall charge will be positive and most dense within the center of the rectangle. We are discussing physics scenarios from the answer.  Students and instructors are in this frame when they discuss physics scenarios or phenomenon, about the properties of physics quantities related to the task at hand. They may also exploit the symmetry of a physical system by investigating related concepts.
The utilization of physics concepts to interpret a physical system. This includes discussing physics scenarios in terms of physics quantities or properties that are relevant to the problem.

Personal similarities and differences:
The five tutorials students worked through have several multi-step problems and include a total of 25 sub-questions. To compare framing of these two students to the same set of questions, we listed the total number of counts for each frame in the table below, as well as counts of each student's first and last frame in a problem. Interestingly, we can see is S1 and S2 have the same percentage of conceptual frames (CP and CM combined, roughly 41%) versus algorithmic frame (AP and AM combined, roughly 59%). Within the conceptual frames, S1 is more likely to be in the CM frame while S2 is more likely to be in the CP frame. Both a chi-square test and fisher's exact test on the frequency distributions of each frame between two students gives a p value <0.05 (=0.036). This personal difference is also manifested in their first and last frames in solving problems. S1 is much more likely to first start with AP frame (start with a physics equation or algebraic expression of some principle, 44%), where S2 always likes to start in CM frame (drawing coordinate system, 60%). This pattern is only suggestive while not statistically significant as suggested by a fisher's exact test on the frequency distributions of the first frame. As far as last frames, S2 is extremely likely to end with CP frame (checking units and making sense of the solution, 80%), and while S2 also has a high percentage of ending in CP frame (50%), they are also likely to end in AM frame (end after the algorithmic math step 32%). Fisher's exact test for last frame distributions between S1 and 2 gives a p value <0.05 (=0.027).

Question characteristics:
It is important not to overlook the similarities despite these personal differences. Even though these two students have different problem-solving habits and preferences, we observe that for a subset of problems, they either start or end the problem in the same frame. We compare the questions when S1 and S2 have overlapping first or last frames. We find when students proceed through problems in similar ways, all of these questions share some common characteristics. First of all, as listed in the table above, out of the 25 first frames, 13 were the same (8 CM, 4AP and 1 CM). Looking at those questions where both students first started in CM frame, all of them specifically ask students to draw coordinate system, or give some information that is relevant to coordinates. When the questions lack these features, students seem to resort back to personal habits where S1 is more likely to first start in AP frame (i.e. start with a physics equation or algebraic expression of some principle, 6 out of 12 times). For those questions where both students started with AP frames, all of them specifically give the mathematical definition about a physical quantity or a physics equation and ask follow up questions; or specifically ask about a physical quantity that would require writing down the algorithmic definition of that quantity. When the questions lack these features, S2 rarely starts in AP frame (once). The correlation between students' first frames and these common characteristics suggest that certain question features can trigger students starting a problem in a certain frame.
Secondly, out of the 25 last frames, 13 were the same (11 CP and 2 CM frames). Looking at those questions where students all ended last with CP frame, all of them share features like asking about unit analysis, the dependence of one physical quantity as another changes, whether the result makes sense, etc. When questions lack these features, students tend to resort back to personal habits. S1 rarely ends in CP frames (twice) but frequently ends at algorithmic math step (AM, 8 out of 12 times). This seems to suggest that certain question features tend to trigger certain last frames.
The preliminary analysis suggests that despite the differences in students' personal habits of problem solving, certain question characteristics could influence students' first and last frames on a problem. When the problem asks explicitly about the coordinates or give some relevant information to coordinates, students tend to start from the conceptual math frame. When the problem asks explicitly about the reasonableness of the answer including unit analysis or the dependence of one physical quantity as another changes, students tend to end in the conceptual physics frame.

V. CONCLUSIONS AND DISCUSSIONS
Students' epistemic framing can play an important role in their ability to solve physics problems. In this study, we explored the relationship between question characteristics and students' framing of a problem. A two-dimensional theoretical framework (CAMP framework: Conceptual Physics, Algorithmic Physics, Conceptual Math, Algorithmic Math) was used to analyze and compare different students' framing to the same set of questions from the CU Boulder tutorials in upper division E&M. After going through training on epistemic framing literatures and applying a coding rubric, two undergraduate students worked through the problems and self-recorded their framing to these questions.
In this preliminary analysis, we compared students' framing on these common questions, including percentages of occurrences of each frame, as well as the distribution of the first and last frames of problems. We found out students' different problem solving habits are manifested in different percentages of certain first and last frames.
However, despite these differences, we found that when these two students do have the same first and last frames, these questions share certain common features. Preliminary analysis suggests there seems to be a correlation between question characteristics and student epistemic framing. In the future, we could use these information to help instructors to better facilitate students' problem solving. For example, given certain question characteristics, instructors would be more aware of the type of frames that students are more likely to be in. Being aware of this can help instructors better guide students and diagnose their difficulty. It also shows these frames can be changed: for student who normally doesn't reflect their answer (CP frame), reminding them explicitly could shift them to such productive frames and form better problem-solving habits.
There are some limitations to this study. First of all, we used self-reported framing of two students instead of external observations. However, using self-reported data relies heavily on research subjects' ability to understand and interpret frames, and on their metacognitive skill to notice their own framing. We gave students' intensive training before the frame mapping process, however, there is still the risk of them not interpreting the training in the same way. However, we find that even though students' frames are somewhat different as they solve the problems, consistent patterns appear when questions share or do not share certain characteristics, supporting our analysis that their interpretation of the framing is probably not random. A possible future step is to triangulate these self-reported results to another set of observations made by experienced researchers; however, previous work has suggested that trying to infer students' frames from their written work alone is difficultwritten work is too sparseand records of students as they are working would be necessary to follow up on these claims. Another limitation of the study is the relative small sample (two students, five tutorials consisting 25 questions). As a first step, we achieved the goal of exploring possible patterns, but in the future, more data on various students would help strengthen these claims.