Impact of Argumentation Scaffolds in Contrasting Designs Tasks on Elementary Pre-Service Teachers ’ Use of Science Ideas in Engineering Design

Yuri B. Piedrahita Urueña, Carina M. Rebello, Chandan Dasgupta, Alejandra J. Magana, N. Sanjay Rebello Department of Curriculum & Instruction, Purdue University, 100 N. University St., West Lafayette, IN, 47907 Department of Physics & Astronomy, Purdue University, 525 Northwestern Ave., West Lafayette, IN, 47907 Department of Computer & Information Technology, Purdue University, 104 N. Grant St., West Lafayette, IN, 47907


I.INTRODUCTION
Over the past few years there have been calls to infuse engineering design concepts and practices into the learning of science at the K-12 level [1,2].Educators and policy makers posit that seeing how science is used to solve real world problems, will increase student engagement and retention.Further, emphasis on science and engineering practices, such as argumentation based on evidence, can prepare them to participate in the 21 st Century workforce.
We have transformed our physics course for future elementary teachers to infuse engineering design into the learning of physics.At the beginning of each instructional unit, students are presented with a design task that they initially approach without being exposed to the necessary science concepts.Then, as they learn various relevant science concepts, they revisit their design task to revise their initial solution based on science ideas they learned.
One of the key challenges in using this approach is that students often resort to trial and error strategies and attend to the superficial aspects of the design, rather than the underlying physics concepts.The tendency is akin to novicelike strategies that students are inclined to use in physics problem solving [3].A vast body of research in problem solving has shown that students often tend to use means-ends analysis and frequently address to the surface features of the problem, instead of to the deep structure.They often search for equations in which to plug in numbers and seldom consider the conceptual underpinnings of the problem at hand [4].Students seem to demonstrate similar novice-like strategies with regard to approaching engineering design tasks.Specifically, they do not attend to the underlying science concepts and often base their design on superficial information of the design task.
We adapted two strategies to address this issue: contrasting cases and argumentation scaffolds.All students were presented with a task to contrast two design solutions.One condition received prompts to identify similarities and differences and produce their own solution, while the other received prompts to produce arguments to support their solution.Our goal was to examine the extent to which different scaffolds affect the conceptual quality of the responses provided by students to the contrasting design task.Our research question is: To what extent do students in each condition: (a) cite scientific ideas covered in the physics class, (b) explain these ideas, and (c) appropriately apply these ideas in their contrasting design tasks?

A. Contrasting Cases
It has been long known that novice students often tend to focus on surface features of problems [4].Contrasting cases is an instructional approach that emphasizes simultaneous consideration of multiple, juxtaposed cases [5].The cases are designed to have embedded similarities and differences which highlight target concepts.They facilitate students to move beyond examining superficial information to instead search for patterns in which underlying principles that are applied [6].By providing students with contrasting cases, they are encouraged to make sense of the problems and consider similarities and differences among them [7].We used an adaptation of contrasting designs in this study.

B. Argumentation
While solving problems, students rarely reflect on the appropriateness of equations or concepts used, and almost never consider alternative solutions [8].Scientific argumentation is a process to rationally resolve questions and solve problems, which shifts the focus from answer-oriented problem solving to process-oriented practice of constructing and justifying claims [9,10].Embedding argumentation in science learning can enhance conceptual understanding and problem solving [11,12].Argumentation can be incorporated into problem solving by asking students to argue how and why they solved a problem [13].A growing body of literature describes argumentation scaffolds and how they can be used effectively [9].Research has found that incorporating justification prompts and questions that emphasize the role of evidence in explanations help college students articulate their problem-solving steps and rationale behind their action [14,15].In this study we use argumentation prompts to facilitate students to create arguments that support their responses to the contrasting case task.

A. Context
This study was conducted in a physics course that enrolled N = 51 future elementary teachers at a large U.S. Midwestern land grant university.The focus of the course was to facilitate students to learn physics concepts in the context of engineering design challenges.This study was conducted at the end of a six-week-long instructional unit on thermal energy transfer.At the beginning of this unit, students were given the task to design a zero-energy home using Energy3D software [16].The software enables students to design a house and measure its energy efficiency.The software allows students to change various design parameters such as the size and shape of the house, the materials used, the size and placement of windows and solar panels, as well as the type and placement of trees around the house.In the first week, students started with a sub-optimal design as a starting point and gradually refined this design.First, they did so based on their prior knowledge and intuitive understandings.Then, the unit led them through a series of hands-on and minds-on activities where each week they learned a different physics concept: radiation, conduction, and convection.Each time they revisited the design and refined it.Finally, in the last iteration, they had to redesign the house from scratch.
In addition to the house design/redesign task, students were also presented contrasting cases of other designs.This was done both in the lecture and the lab.Students had to critique the designs that were being contrasted.
Each section of the class was assigned a condition: control or experiment.Both completed the same design challenge, hands-on laboratory activities, and contrasting design tasks.The control condition (NC = 30) was asked to provide an explanation to support their answers.The argument condition (NA = 21) was provided the 'Criteria for Good Scientific Arguments' shown in Fig. 1.For all written work, the argument condition was required to support their answers following the criteria for good scientific arguments.
Use the criteria below for good scientific arguments when you write your responses or discuss your ideas in groups.

C. Contrasting Design Task
Data were also collected in a contrasting design task which was part of the quiz at the end of the unit.Students were presented with a contrasting case task focusing on the design of a solar cooker.This transfer task used the same principles as the design of an energy efficient house, but was a different context.The task is shown in Fig. 2. Each condition completed the same task, but was provided a different set of question prompts as shown in Fig. 3.
You are taking a group of elementary children on a field trip through the woods on a cool, sunny fall day.Around noon you decide to take advantage of the sun to heat up some soup you have brought with you.You have the following materials: tin cans, aluminum foil, black trash bag, transparent zip lock bags, Styrofoam cups.Two students come up with two different designs for how to use the materials to build a 'solar cooker' -a device that will heat up the soup.Jose: I will put the soup in the Styrofoam cup because this will not allow heat to escape.I will take the aluminum foil and wrap the cup on the outside because it will prevent the cold from outside from coming in and cover the top with a cut out piece of black trash bag and cover the top of the Styrofoam cup so that it can absorb the sun's heat.
Jamila: I would prefer to use the tin can, because the tin allows the heat to come in easily.I will cut out a piece of the black trash bag material and wrap it on the outside because black absorbs heat from the sun.I will not cover it with anything, because it you cover it, you block out the heat from the sun.After it has heated up some, then I will cover the top with foil.
Which explanation (or combination thereof) best provides a justification to design a 'solar cooker'?Or, do you have a different explanation.Explain, elaborate, and justify your preferred explanation.Students' responses were coded in terms of the scientific ideas 'cited', 'explained', and 'applied'.A scientific idea was defined as one that students were expected to have learned through the activities they completed in this unit.A response that 'cited' an idea mentioned the term or label for the concept.A response that 'explained' a science idea provided a simplified description of the concept, perhaps in terms of other concepts.A response that 'applied' a scientific idea demonstrated how the concept related to the particular context of the design solution.In our coding scheme, an idea that was applied had to be explained, and an idea that was explained had to be cited.Table I shows an example of response fragment that cites, explains, and applies the concept of "albedo."The coding was done by the first author.An inter-rater reliability of 100% was established after discussion with the secondary coder, co-author (NSR).

Response Fragment
Analysis: Concept… "The black trash bag has low albedo, so more heat will be absorbed rather than reflected.This will assist in heating." Cited: "low albedo" Explained : "more heat will be absorbed rather than reflected" Applied : "The black trash bag … will assist in heating"

A. Pre/Post-Test
A t-test, paired two-sample for means, of the pre/post-test data together for both conditions (N = 51, NC = 30, NA = 21), showed a statistically significant increase (t(44) = -12.78,p < .001)from pre-test (27.316.15) to post-test (36.334.53)scores.There were no statistically significant differences between the control and argument conditions on either the pre-or post-test.

B. Contrasting Design Task
We identified 10 distinct science ideas (See Table II) embedded in students' responses to the contrasting design task.Additionally, we coded examples of conduction, convection, and radiation, as science ideas.We compared the two conditions with regard to the number of different scientific ideas 'cited', 'explained' or 'applied' in their solutions independently as well all the three together.The trends were similar because the 'apply' category subsumes 'explain', which in turn subsumes 'cited'.We show the results of the analysis completed together (See Fig. 4).A single factor ANOVA showed a statistically significant difference between the two conditions (F(1, 49) = 32.961,p = .001),with the argument condition citing, explaining, and applying scientific ideas significantly more frequently than the control condition.

V.DISCUSSION AND CONCLUSION
A common challenge to leveraging the use of engineering design to facilitate the learning of science, as identified in recent reform documents [1,2], is that students often tend to use a trial and error approach.That is, students often circumvent the underlying science principles to complete the design tasks.We implemented two strategies in a physics course for future elementary teachers to address this issue.The first was the use of contrasting cases that would provide students examples of design solutions to critique.The second was the use of argumentation scaffolds that would provide guidelines for the critique of the contrasting designs.All students received the contrasting design task, but only one condition received argumentation scaffolds.Thus, our study investigated the effectiveness of argumentation scaffolds in the context of a contrasting design task.
We found no significant differences between the two conditions with regard to their gain scores on a multiple choice assessment.This demonstrates that no condition had a greater understanding of the concepts per se.However, in comparing their responses to the contrasting design task, we find that the argumentation condition cited, explained, and applied science concepts much more frequently than the control that did not receive and was not prompted to use the guidelines for good scientific argumentation.
The results of this study seem to suggest that the use of contrasting designs together with argumentation scaffolds used here, may be a promising intervention to facilitate students to science concepts in engineering design tasks.A deeper analysis is needed to identify differences in the quality of students' ideas (e.g., citing vs. explaining vs. using).Research is also warranted to investigate whether the aforementioned advantage transfers to tasks that require students to come up with their own design solutions rather than simply critique two or more contrasting designs.Similarly, our work will also investigate differences in students' design approaches as well as their design solutions.

FIG 4 .
FIG 4. Science ideas used in the Contrasting Design Task.

. Pre/Post-Test The
• Claim: do you clearly state your position on the issue?• Evidence: do you support your claim with evidence from a theory or observation that all can agree on?• Reasoning: do you provide single of multiple ways to explain how your evidence supports your claim?• Counter position: have you fairly presented alternative positions and counterarguments with supporting reasons?pre/post-test consisted of 45 multiple choice questions on heat transfer, taken from MCAS (Massachusetts Comprehensive Assessment System) from 2010-2015, Spring Release, strands of Introductory Physics and Technology/Engineering for High School Courses, and the topic of Heat and Heat Transfer [17].The test was administered at the beginning and end of the unit.

TABLE II .
Science Ideas Used in Responses