Student sense-making on homework in a sophomore mechanics course

When students solve physics problems, physics instructors hope that they use and interpret algebraic symbols in coordination with their conceptual understanding, their understanding of geometric relationships, and their intuitions about the physical world. We call this process physics sense-making. “Plug-and-chug” and “template” problem solving strategies, which are common for many students, exclude sense-making. We have designed a mechanics course for sophomore, undergraduate students that emphasizes sense-making and traditional physics content in equal measure. Sense-making is supported in all aspects of the course: during in-class activities, on augmented homework assignments, and on exams. While sense-making prompts on homework assignments are strongly scaffolded at the beginning of the course, these supports fade as the course progresses. In this paper, we discuss an analysis of students’ homework responses to open-ended sense-making prompts throughout the course.

Physics instruction seeks to support the development of future physicists by cultivating expert-like problemsolving skills in students.Experts commonly use various sense-making strategies when solving physics problems but students rarely do [1].Sense-making is the use and interpretation of algebraic symbols in coordination with one's conceptual understanding, their understanding of geometric relationships, and their intuitions about the physical world."Plug-and-chug" and "template" problem-solving strategies, which are common for many students, exclude sense-making [2,3].
Lenz and Gire have found that while professors believe reflection-based sense-making abilities are important skills for students to develop, they do not explicitly teach theses strategies in their courses [4].Little is known about how explicit instruction of sense-making strategies affects student performance of these strategies; Warren's work with algebra-based physics and evaluation strategies is one example [5].This paper bridges the gap between instructional goals and student development of these sense-making skills.We do this by putting forth an analysis of data collected from a middle-division physics course that specifically augments traditional homework (focused on physics content) with explicit sense-making prompts to help students mature their sense-making abilities.

II. SENSE-MAKING IN THE TECHNIQUES OF THEORETICAL MECHANICS COURSE
The data for this study were collected from a new physics course, Techniques of Theoretical Mechanics, at Oregon State University, a large, public, research intensive institution.This course was created and taught by author EG to help students make the transition between introductory and upper-division physics through an emphasis on sense-making.
The physics topics of the course are: finding equations of motion with Newton's Laws by solving differential equations, velocity-dependent drag forces, rockets with variable mass, Lagrangian and Hamiltonian tech-niques, and special relativity.The class met for 50 minutes 3 times per week and small-group problem-solving activities happened at least once per week.
The prerequisite courses are multivariable calculus and the first 2 quarters of introductory physics (which include Newtonian mechanics for translational and rotational motion and waves).Of the 27 students who completed the course, 3 were co-enrolled in the last quarter of the introductory physics sequence, 11 students had completed the introductory sequence the previous quarter, and 5 students had already taken most of the junior-year Paradigms courses [6].
The theme of this course is physics sense-making.The sense-making goal was treated on equal footing as the physics content goals of the course: it appeared on the syllabus, was discussed in almost every class meeting, appeared in every homework problem, and was included on exams.During in-class small-group problem-solving activities, the instructor asked for several volunteers to come to the board to demonstrate a sense-making strategy they used for the problem.During the second week of class, while considering a Newtonian mechanics problem, the class brainstormed a list of strategies that could be used to check the correctness of the answer of the problem (Table I).This list was posted on the course website for reference.We describe how sense-making was included on homework in Section III.Sense-making was primarily discussed in terms of evaluation strategies: strategies for evaluating the correctness of an answer at the end of a solution (and at intermediate steps).Some discussion in the course focused on sense-making at the beginning of a problem to orient oneself, but this discussion was less formalized than the 2017 PERC Proceedings, evaluation strategies.We intend to make this orientation aspect of sense-making more formal in the future.

III. METHODS
Students were assigned 10 weekly homework assignments during the 10 week term.The first 8 assignments contained classical mechanics problems and the last 2 assignments contained special relativity problems.The coursework was structured to support students' sensemaking development as well as provide multiple lenses into students' application of these strategies.
Each assignment included 2-4 problems with 2-10 explicit sense-making prompts embedded in the problems.This course was designed to implement Rosenshine's scaffolding and fading approach to teaching higher-level cognitive strategies as a way to aid students in learning to utilize sense-making strategies [7].Thus, the first 3 homework assignments had sense-making prompts that directed students to use specific sense-making strategies (e.g. after finding an equation for the range of a projectile on an incline: "Sense-Making: Consider Special Cases Does your result for the maximum range make sense if the ground is horizontal?If the ground is vertical (like right up against a cliff)?").On Homework 4-7, the prompts for sense-making were intentionally faded; they became less prescriptive and more open-ended (e.g."Find the equation of motion (acceleration) of the bead.Use at least two sense-making strategies to make sense out of this equation.").On Homework 8, the sense-making prompts were faded yet again; they did not specify a particular number of strategies (e.g."Be sure to do some sense-making around your result").The sense-making prompts became more specific again for the last 2 homework assignments which featured special relativity problems.Twenty-nine students turned in homework during the term.Student solutions were scanned twice: before grading (for a clean copy of the students' work) and after grading (so we could record the feedback students were receiving about their sense-making performance).
Students were provided with written feedback on the content of both their solutions to the physics problems and their responses to the sense-making prompts.The feedback took the form of short questions aimed at drawing the student's attention to places where errors occurred and asking them to consider what changes they might have made to their solution or how they displayed their work and their reasoning.Homework was promptly graded and returned to students, typically within 1-2 class days.Often, the grader gave a brief announcement to the course as a whole identifying common errors on the assignment and suggesting how students might improve for future assignments.Detailed solutions to the homework, including responses to sense-making prompts, were made available on the course website.
In this paper, we report on the sense-making strategies students used on Homework 4-8 where they were asked to use sense-making strategies but these strategies were not prescribed.These assignments contained a total of 15 sense-making prompts: 12 that specified the number of strategies to use and 3 that did not.We received assignments from 27 study participants, although not all students responded to every sense-making prompt.
Many responses contained several sense-making strategies, and each strategy was individually coded using an emergent coding scheme [8].Although it was generated independently, unsurprisingly the strategies found were similar to the student generated list found in Table I.Often, students labeled which strategy they thought they were using.We coded the students' work based on what the student actually did, thus our codes sometimes differed from how the student labeled the strategy.

IV. RESULTS
The codes of student work and their accompanying descriptions are presented in Table II.In the end, the data set contains 333 responses to sense-making prompts; out of those came 825 coded sense-making strategies.During coding, 18 unique sense-making strategies were identified (Table II).These strategies were broken into 3 categories: dimensions, cases, and other strategies.Many of these strategies aligned with those identified by the students (Table I) though not all did.Examples of students' work demonstrating some of the most frequently used sensemaking strategies can be found in Fig. 1-6.
The most common sense-making strategy was to check the units or dimensions of an answer.Students performed this strategy in a number of different ways.By far the most common way was to substitute the fundamental dimensions of quantities (e.g.length, mass, and time) into the answer equation and then check that the dimensions on both sides of the equals sign were the same (Fig. 1).
FIG. 1. Student example of using fundamental dimensions to analyze the Lagrangian of a free particle.
We also observed students perform this process with (1) units (e.g.meters, grams, and seconds) instead of fundamental dimensions or (2) compound dimensions (e.g.acceleration, force, and energy) (Fig. 2).The strategy of using compound dimensions was advocated for by the instructor on the second day of class as having 2 advantages: (1) it can be faster than breaking everything down into fundamental dimensions, and (2) it fosters deeper understanding of the connections between quantities.However, we found that students infrequently used this compound dimension strategy when checking dimensions on their homework solutions.
The next most common sense-making strategy was to check special or limiting cases.The special case strategy is when a student evaluates their equation-answer with precise values that allow for meaningful interpretation/comparison (Fig. 3).A limiting case strategy is similar to special case but requires a student to take the limit of the equationanswer as a particular variable goes to a specific value, typically 0 or ∞ (Fig. 4).When labeling their homework solutions, students often did not distinguish between these 2 strategies and often called a limiting case a "Special Case."This led to many instances where our codes differed from the students' labels.Another common sense-making strategy was conceptual connection (Fig. 5).When using this strategy, students explained why an answer made sense in terms of their conceptual understandings of the physical situation.Another frequently used strategy was the functional dependence strategy.A student using this strategy com-mented on whether the behavior of the function in an equation-answer makes sense for the physical situation.The students often called this strategy "Proportionality" from their list of strategies (Table I).Using this strategy, a student might comment on whether (a) the answer is expected to depend on a particular physical quantity, (b) the answer should increase or decrease when a physical quantity is varied, or (c) if the qualitative behavior of the equation-answer matches the expected behavior of the physics system, such as having a maximum value or oscillatory behavior.Students often confused this strategy with limiting case, mislabeling one as the other.

V. LIMITATIONS
The students analyzed are primarily physics majors at a large, public, research institution.While students are required to turn in individual work they are encouraged to work together.An individual's assignment may not be entirely representative of individual thought; due to the nature of homework and this encouragement to work together the collected data is a polished version of student reasoning.While this study was conducted in a course with explicit attention to sense-making strategies, the differences in teaching techniques from traditional teaching techniques are not fully documented.We make no new claims about student reasoning in a course that does not emphasize sense-making.This paper focuses solely on analysis of written work, therefore the coding of strategies does not reflect students' evaluation process.

VI. DISCUSSION AND IMPLICATIONS
Students were found to gravitate towards fundamental dimensions, special case, conceptual connection, and functional dependence as their primary sense-making strategies.Not all of these strategies were intended to be emphasized as the best strategies in course instruction.Specifically, checking units/dimensions was considered to be "low-hanging fruit" and students were told that while it is always good to do, it alone is not enough.Thus it is not surprising that students heeded this advice and checked units/dimensions on the majority of their work, often as a first step before further sense-making.
Furthermore it is unsurprising that special case and conceptual connection were prevalent strategies, especially due to student's comfort with the physical connection of the classical mechanics material.Students were most likely comfortable interpreting the results of special case analyses and able to draw on their experience with introductory physics and the physical world to make conceptual connections with their solutions.
Rosenshine's scaffolding technique was implemented to cultivate a course-wide norm of sense-making [7].The effectiveness of this scaffolding was analyzed through students' choices of sense-making strategies on the homework.While this scaffolding worked well for many strategies, students did not choose to use some of the strategies that were emphasized in the course.Two such strategies are: using power series expansions to understand a solution and predicting the form of the solution on the beginning of problem solving.While these strategies do not possess the widespread applicability of fundamental dimensions they are useful strategies that are applicable to the homework prompts analyzed here.We value the use of these strategies and future instruction will be tailored to increase their emphasis.
Other changes that will be made for future instruction will be: emphasize the distinction between a special case and a limiting case and how to choose advantageous cases to analyze.While students often used these strategies, they did not always choose cases that would yield the most insight into the problem.Lastly, future instruction will expand the sense-making emphasis from a reflectioncentered approach to include using sense-making as a means of orienting oneself to the problem.Despite these intended modifications, this approach of having the sense-making goal treated on equal footing as the physics content goals of the course is a promising approach to support middle-division students in making sense of physics problems like experts.For instructors who believe that sense-making goals should be more explicit, we found our scaffolded approach to be highly successful.In particular, the combination of an interactive course environment and the comprehensive inclusion of sense-making in class meetings, homeworks, and exams proved highly successful at promoting sense-making.
edited by Ding, Traxler, and Cao; Peer-reviewed, doi:10.1119/perc.2017.pr.035Published by the American Association of Physics Teachers under a Creative Commons Attribution 3.0 license.Further distribution must maintain attribution to the article's authors, title, proceedings citation, and DOI.

FIG. 3 .
FIG. 3. Student example of using special case to analyze an equation of the velocity as a function of mass for a rocket with linear air resistance; as seen by them setting m = 0.

FIG. 4 .
FIG. 4. Student example of using limiting case to analyze an equation of the velocity as a function of mass for a rocket with linear air resistance; as seen by them taking m → 0.

FIG. 5 .
FIG. 5. Student example of using conceptual connection to analyze the constraint force (λ), found through undetermined Lagrange multipliers, of a particle confined to the surface of a cylinder.

FIG. 6 .
FIG. 6.(a) Equations of motion (b) Student example of using functional dependence to analyze the equations of motion seen in part (a), of a spherical pendulum.

TABLE I .
List of sense-making strategies generated by students during Week 2.

TABLE II .
Sense-making strategy codes, descriptions, and frequency (% of 825 total code applications).
compares answer to real world experiences or knowledge from a previous course 7% Sign checks that the sign of the answer makes sense with their coordinate system 3% Visualization understands the solution through figures, diagrams, graphs, etc. 2% 2nd Way compares answers using 2 solution methods 2% Algebrastates that the answer is correct because the algebra was done correctly 1% Assumptions checks for consistency between the answer and assumptions made at the beginning of the solution<1%Reasonable Magnitude states or argues why the magnitude of the answer is reasonable <1% Authority <1% Strategy Identification identifies potential sense-making strategies but doesn't implement them <1% No Sense-Making does the problem but does not answer the sense-making prompt 3% FIG.2.Student example of using compound dimensions to analyze the Lagrangian of a free particle.