How prompting force diagrams discourages student use of adaptive problem-solving shortcuts

Current step-by-step, problem-solving frameworks have been shown to improve aspects of students’ problem solving. However, conceptual problem-solving shortcuts that break from these frameworks may demonstrate adaptive problem solving. Following an approach from Heckler [1], we show that prompting introductory physics students to complete a common initial step of these frameworks (“draw a free-body diagram”) before solving force problems decreases the use of conceptual shortcuts and increases reproduction of standard procedures. Extending this paradigm, students were then asked to evaluate a sample informal solution for a force problem. When the problem text included a diagram prompt, students rated the informal solution less favorably, commenting that the solution was not formal enough. These results imply that the prompt to draw a diagram not only cues procedural problem-solving approaches, but also may push students away from informal approaches that demonstrate problem-solving expertise.


I. INTRODUCTION
Student use of step-by-step problem-solving frameworks has been shown to increase problem-solving performance in many areas, including: use of appropriate diagrams, identification of relevant concepts, and correct final answers [e.g.[2][3][4][5].These frameworks, developed for introductory physics problem solving, typically require the construction of a diagram, selection of an appropriate physics concept, application of the concept to the specific problem, and evaluation of the final solution.To help students, such frameworks structure problem solving into standard, procedural approaches.
Yet, one consequence of these standard procedures is the lack of support for more informal solutions that can demonstrate problem-solving expertise.For example, one aspect of problem-solving expertise not valued by standard procedures is the opportunistic application of conceptual shortcuts that simplify or eliminate explicit calculations [6].Such conceptual shortcuts involve reasoning flexibly about each problem rather than following a scripted set of steps.
Here, we show that cuing a typical element of problemsolving frameworks -namely, the drawing of free-body diagrams during problem solving -increases the use of more standard problem-solving procedures, while discouraging use of more informal, conceptual shortcuts.

II. A CONCEPTUAL SHORTCUT FOR SOLVING A FORCE PROBLEM
Consider the following problem, from Heckler Heckler [1] observed a formal solution to this problem, which corresponds to many problem-solving frameworks.In this solution, students draw a sketch of the situation along with a free-body diagram for each of the three boxes containing Liz, Dan, or Rex, as shown in Figure 1.

FIG 1.
A sketch and free-body diagrams for Q1.
Following the framework, students would then select Newton's 2 nd law and write for each box.To find the tension in the rope between Liz's and Dan's box, only the Newton's 2 nd Law equations from Dan's and Rex's boxes, with the fact that a = 0, are needed.Using these two equations (T 1 = T 2 + f from Dan's box and T 2 = f from Rex's box) to eliminate T 2 , these students find that T 1 = 2f = 2μ s mg = 120 N. We refer to this approach as the Separate Boxes approach.
Heckler observed another possible solution to Q1, shown in Figure 2.This solution combines Dan and Rex into one unit.At constant speed, the tension between Liz's and Dan's boxes equals the friction on the "Dan+Rex" unit, which is twice as heavy as a single box.Calculating the values, the tension is 120 N, the same correct answer obtained using the Separate Boxes approach.We refer to this solution method as the Combine Boxes approach.Although both approaches lead to correct solutions, the Combine Boxes approach leverages a particular conceptual insight that simplifies the mathematical manipulations for this particular problem.Instead of mathematically combining the expressions for Dan's box and Rex's box, this solution preempts that need by conceptually combining the two boxes.By contrast, the Separate Boxes solution is more computationally complex.However, it is also more general because the strategy of drawing the individual freebody diagrams and writing the Newton's 2 nd law equations for each box can lead to success in solving for any of the unknown values in this problem.Combining Dan's and Rex's boxes is useful in solving for the particular quantity asked for in this problem, but not others (such as finding the tension in the rope connecting Dan and Rex).Because the advantage of combining boxes is problem-dependent, this type of shortcut is not well prescribed or assessed by problem-solving frameworks that aim for general applicability [7].

III. STUDY: THE EFFECT OF DIAGRAM PROMPTS
Consider the standard prompt asking students to draw a free-body diagram before solving force problems.On the one hand, this prompt could help focus students' problem solving with productive problem-solving procedures, increasing problem-solving success.On the other hand, use of these procedures may discourage efficient, conceptual shortcuts.
For Part A of our study, we adopted a paradigm from Heckler [1], manipulating whether or not students were directed to draw a free-body diagram before solving force problems.Studying science and engineering majors in introductory physics courses, Heckler found that such freebody diagram prompts designed to aid students in solving the problem in fact pushed students away from intuitive solutions that demonstrate sense-making.Our results replicate Heckler's finding.Additionally, we find that these intuitive conceptual shortcuts can be more likely to lead to correct answers than standard procedural approaches.
In Part B of our study, extending Heckler's original paradigm, we asked students to evaluate the quality of a sample informal solution.Students who received a diagram prompt rate the quality of this informal solution lower than students who do not.This implies that the diagram prompt leads students to devalue informal approaches.
The data were taken from a large-lecture, introductory physics course in mechanics geared for biology, social science, and pre-med students.The data collection occurred during the first 20 minutes of the final discussion section.Students worked individually on an ungraded handout containing the questions as review for the final exam.

A. Effect of diagram prompt on problem solving
In each condition, students solved two force problems, Q1 (the "train" problem described earlier) and Q2 (a problem where a "train" of two cars is being pulled up a ramp).Students in the Prompt (P) condition were told to draw a free body diagram for the middle box of the train in Q1 and for each box in Q2 before being asked to solve the problem.Students in the No Prompt (NP) condition were simply asked to solve the problems.Condition was assigned randomly to control for students' prior knowledge and motivation, with half of the students in each discussion section assigned to either condition (n NP = 71, n P = 69).
Importantly, both Q1 and Q2 could be solved correctly with either a more procedural Separate Boxes approach or a more informal Combine Boxes approach.

Results
Almost all students in both conditions drew some sort of force diagram: 90% of NP students in Q1 and 93% of NP students in Q2 drew a force diagram, while all P students drew force diagrams on Q1 and Q2.The influence of the prompt was on the type of diagram drawn.In the P condition, 94% and 93% of students drew only diagrams representing the separate boxes of the problem, on Q1 and Q2 respectively, compared to 55% and 49% of NP students.
The diagram prompt influenced students' approaches to solving the problems, shown in Figure 3. Compared to students in the P condition, students who were not prompted to draw a force diagram were more likely to use a Combine Boxes approach on both Q1, χ 2 (1, N=137) = 4.99, p = .026,and Q2, χ 2 (1, N=128) = 9.49, p = .002.Students prompted to draw a free-body diagram tended toward more standard, procedural problem-solving approaches.

FIG 3.
The breakdown of problem-solving approaches used on Q1 and Q2.For both questions, NP students used the Combine Boxes approach more and the Separate Boxes approach less than P students.
Looking at correctness, a higher percentage of students who used Combine Boxes found correct answers, as compared to students who used the Separate Boxes approach (see Table 1).This pairwise difference is significant for Q1, χ 2 (1, N=99) = 4.19, p = .041,but not for Q2, χ 2 (1, N=104) = 1.17, p = .28.Approaches coded as "Other" (e.g.conservation of energy) were used equally across conditions and led to a very low number of correct solutions.Taken together, these results show that the diagram prompt shifted students away from a conceptual shortcut and toward a more methodical, procedural approach.Students who used a conceptual shortcut bypass lengthy calculations without sacrificing correctness.

B. Effect of diagram prompt on solution rating
Following these problems, students evaluated a sample solution to a new problem, taken from Heckler [1]: Mary Kate is pushing on a box with a force of 480 N in one direction and Ashley is pushing the box with a force of 340 N in the opposite direction.The box is not moving or beginning to move.There is friction between the box and the floor, and the coefficient of static friction is μ s = 0.4 and the coefficient of kinetic friction is μ k = 0.25.

• [For Condition P only: Draw a free body diagram clearly indicating the forces.]
• What is the minimum mass that the box can be in order for it to remain motionless?Show your work.
The informal sample solution conceptualized the problem in two steps.First, a diagram depicted the forces of only the two sisters, with "F net = 140 N" written below it.Then, a second diagram depicted the net force of the two sisters opposing static friction, the equation was written out, and m was solved for correctly, giving the minimum required mass.This "two-step intuitive solution" was illustrated by Heckler's original study.Students were asked to rate this sample informal solution on a 5-point Likert scale (1: "very bad" to 5: "very good") and to explain their rating.This solution was chosen because it represented correct reasoning but used informal diagrams and an informal solution approach.To distinguish between these two aspects, we coded whether each student's explanation explicitly cited a violation of a formal solution approach, either pointing out that F net on a static object should equal 0 and not 140 N, or that the formal Newton's 2 nd law equation should be written explicitly.

Results
Students' mean ratings of the sample informal solution are shown in Figure 4.When the problem text included a diagram prompt (P), students ranked the solution 0.4 points less satisfactory on average, t(100) = 2.30, p = .024.

FIG 4.
Mean ratings of the sample informal solution (1: "very bad" to 5: "very good").Error bars are 1 SE of the mean.
To understand the differences in ratings across conditions, we analyzed students' written explanations for their ratings.We compared the explanations given by students who used a Separate Boxes approach on both Q1 and Q2 ("Formal Solvers") to students who did not ("Not Formal Solvers").Figure 5 shows that Formal Solvers in both conditions were more likely to cite violations of the formal approach.Among students who were not formal solvers, those who saw the diagram prompt (P) were more likely to mention a violation to the formal approach.Backwards-elimination log-linear analysis shows that the interaction of condition with prior approach is significant, χ 2 (1, N=94) = 4.63, p = .031.

FIG 5.
The percentage of students who cite a lack of formality in their sample solution ratings, broken out by whether or not they were Formal Solvers on Q1 and Q2.
Our interpretation of these results is that students who only used the Separate Boxes approach previously ("Formal Solvers") expected this type of formal approach to be appropriate when evaluating a sample solution, independent of a diagram prompt in the problem text.For students who did not exclusively use procedural, formal approaches, the presence of the diagram prompt increased the rejection of correct, informal solutions by cuing one aspect of standard procedures.Importantly, a prompt to draw a diagram appears to not only suggest that formal diagrams should be drawn, but also that formal solution approaches should be taken.

IV. CONCLUSION
The results from Part A indicate that prompting freebody diagrams before problem solving leads to a small but significant decrease in the use of conceptual shortcuts that simplify mathematical computations, replicating one of Heckler's original findings.Thus, a common instructional scaffold may inadvertently discourage students from intuitive conceptual shortcuts that demonstrate adaptive problem solving.We also found that these conceptual approaches were, on some problems, more likely to lead to the correct answer, a finding potentially consistent with Heckler's finding that students who did not receive a diagram prompt were more likely to get the answer correct.Furthermore, Part B extends Heckler's original paradigm to show that the diagram prompt leads students to find fault with correct, though informal, reasoning.
Prompts suggesting adherence to procedural problemsolving frameworks can provide structure for helping introductory students develop problem-solving competence.Yet, these results suggest that if students remain beholden to these procedural frameworks, they may be less likely to reason flexibly about each problem, and, as a result, be less likely to capitalize on opportunities to apply conceptual problem-solving shortcuts.The key question for future study is how to incorporate adaptive problem-solving strategies into existing instructional practices emphasizing standard procedures.
Another open question is whether alternative diagram prompts would have a different effect.For example, future research could investigate whether a suggestion to draw any helpful diagram, instead of a requirement to draw a specific free-body diagram, could preserve the benefits of such prompts while mitigating the rote activation of standard procedures.
This study speaks to an ongoing discussion in the field about what we value as researchers and instructors, and, in turn, what students learn to value.We suggest that researchers and instructors should use procedural frameworks cautiously to support the development of problem-solving competence without discouraging students from making use of emerging problem-solving expertise.
[1]: Q1 ("Train"): Three siblings, Margaret, Dan, and Liz, are playing in the basement.With some rope, they attached three boxes together in a line like a train.Liz sits in the first box, Dan in the second, and they put the dog Rex in the third box.Margaret grabs on to the first box and pulls the "train" around the basement.When the kids (and the dog) are sitting in their box, each box has a total mass of 30 kg, and the coefficient of friction for the boxes on the basement floor is μ k = 0.2.At one point, Margaret is pulling horizontally and the "train" is moving with constant velocity v = 2.0 m/s on the level basement floor.With how much force is the rope from Liz's box pulling on Dan's box?Show your work.

FIG 2 .
FIG 2. A student's Combine Boxes approach to Q1.The key feature is that Dan and Rex are treated as one combined unit.

TABLE 1 .
The percentage of correct solutions produced by different solutions approaches.