Reading between the lines : lab reports help high school students develop abilities to identify and evaluate assumptions

Most of the research surrounding development of scientific abilities among introductory physics students has been conducted at the college level. A shift in the high school science culture encouraged by NGSS has changed the focus of K-12 physics education making similar work necessary at the K-12 level This paper describes how high school students in a first-year physics course developed assumptionassociated abilities during the six months they studied mechanics. Using their written laboratory reports, we found that repeated exposure helped students gain competence identifying and evaluating assumptions. While it took high school students longer to demonstrate proficiency than college students, by the end of the mechanics portion of the curriculum, 90-95% of students achieved proficiency or partial proficiency.


I. INTRODUCTION
The American Association of Physics Teachers (AAPT) recommends fostering the development of 21 st century skills and competencies in undergraduate physics laboratories [1].The Next Generation Science Standards (NGSS) promote the development of science practices in the K-12 setting [2].How do students develop recommended skills, competencies, and science practices in the context of a physics curriculum?
Student development of scientific abilities has been studied at the college level [3].Research and assessment tools described in references 3-7 help study student engagement experimental design, data analysis, evaluation of uncertainties and assumptions, and communication in introductory physics courses [3][4][5][6][7].Assumptions-related studies found these abilities to be difficult for college students.We decided to investigate how high school students develop such abilities.Although there have been studies examining high school student learning of authentic scientific practices, very few focus on assumptions [8,9].
Although high school students are younger than college students, unlike college students, they engage in physics lessons with the same teacher every day.This provides repeated opportunities for the students to develop targeted scientific abilities and for researchers to track their development with greater frequency.During the 2015 -2016 academic year, we collected and scored laboratory reports of first-year high school physics students in an effort to better understand how they learn to identify and evaluate assumptions.

II. PRIOR WORK
The shift in high school science culture encouraged by the NGSS has changed the focus of a K-12 physics classroom from prescribed processes and procedures to exploratory and evidence-based learning.High school students engage in practicing science by working collaboratively through the design and implementation of investigations [10].Using the Investigative Science Learning Environment (ISLE) framework, students can learn physics by developing ideas with their peers through a series of observational, testing, and application experiments [11].The goal of this repeated engagement in authentic scientific practices is to help students develop scientific abilities [12].The term "scientific abilities" refers to the processes, procedures, and methods that scientists use when constructing knowledge and solving problems [3].
Among the many scientific abilities that Etkina and colleagues identified and studied were the abilities to identify and evaluate assumptions -factors to be assumed true when applying a certain mathematical model to a real world situation [4].For example, when we say that air does not affect the flight of a projectile -this is an assumption.Another term that is used when physicists talk about assumptions is "systematic errors," or correctable/adjustable factors involved in conducting the experiment itself [13].For example, when describing the motion of a cart moving across the floor, students often assume but do not actually check that the surface is leveled before beginning the experiment or conducting an analysis of the data.
One way we can study student development of scientific abilities, specifically those associated with assumptions, is through written lab reports.Prior research at the college level shows it takes on average 5-8 weeks for 80% of the students in a class to become proficient or partially proficient in a specific ability.However, some abilities take longer to develop and fewer students become proficient in them even by the end of the semester.Specifically, the most difficult ability that takes the longest time to develop is the ability to evaluate the effects of assumptions: fewer than 75% of students become proficient or partially proficient in it by the end of the semester [12].Abilities associated with assumptions have not been studied at the high school level.In the case study described here, we examine how high school students develop assumptions-associated abilities.

A. Setting
The case study was conducted with high school students in a first-year honors physics course in Central New Jersey suburban school district.The student population is a mix of Caucasian, African American, East Asian, and Hispanic students.The course followed the Physics Union Mathematics (PUM) curriculum (http://pum.rutgers.edu)and used ISLE-based laboratories developed by the Rutgers University Research Group.Students in ISLE labs design their own laboratory investigations and evaluate their progress through the use of scientific ability rubrics [3].To learn new procedures and new assumptions-related vocabulary at the beginning of the school year, students were engaged in a series of whole-class ISLE-based investigations where they were asked to make assumptions.The classroom teacher (first author of this paper) provided scientific ability rubrics to the students at the beginning of each laboratory investigation and then used the rubrics to provide feedback on student reports.The rubrics for assumptionsassociated abilities are located in the Appendix.

B. Data sources and participants
The data for the study came from students' written work.In class, students used white boards and notebooks to record data.They then wrote their laboratory reports in Google Documents and submitted them through Google Classroom.These Documents became the data for the study.For each report, students received a teacher-created template document shared using the Google Classroom software.This template did not provide students guidance in writing their report; it only contained a designated space for reflection and a table for self-assessment.After feedback, the students had an opportunity to revise their work and demonstrate an improved understanding.During mechanics, students wrote 8 formal reports.
We examined lab reports for approximately seventy physics students over a six-month time period.Students worked in groups of three or four and then wrote reports either individually or collaboratively.Laboratory groups changed approximately once a month so students had the opportunity to collaborate with all of their peers.

C. Data collection and analysis
For the purposes of this case study, the unit of analysis was one lab report.We collected a total of 246 reports for the study and organized the reports by student or student group and lab assignment and then evaluated them using the scientific ability rubrics.Raw scores were then entered into an Excel file for analysis.To ensure that scores for assumption-associated abilities were reliable, we used the following procedure.A second reader other than the classroom teacher scored 10 laboratory reports using the rubrics.Then we discussed the discrepancies in the scores.The second reader then scored an additional 10 laboratory reports and we achieved 90% agreement.
The findings section presents a detailed analysis of student progress toward proficiency.

IV. FINDINGS
To learn how high school students develop assumption-associated abilities, we first examined the first version of submitted reports at different points in the mechanics portion of the curriculum.Then, we looked at student progress over time to determine growth.
Two weeks into the school year, the students conducted their first ISLE-based laboratory.They had to design an experiment to predict where two batterypowered cars would meet when placed a certain distance apart.In groups, they first designed an experiment to determine the speed of each car and then had to come up with a model that would describe its location at any point in time.The students then used their model to predict where the two cars would meet.Upon completion of the investigation, they described their findings in reports submitted through Google Classroom.
Below we show excerpts from one of the best reports.The student included the following assumptions in her report.She assumed that "…the floor was smooth and flat (no incline), the car ran parallel to the measuring tape we used…and the car would travel at a constant speed once set into motion."After stating her assumptions, she went on to explain the effects of the assumptions and how her group validated them.For example, "to test our assumption that the floor was flat, we put a marble on the floor, and observed if it was either stationary or moving.The marble generally remained stationary, so this helped to support our assumption that the floor was flat."Her identification and evaluation of assumptions earned a score of proficient on the scientific ability rubrics.Not all students were as successful writing their first reports.
After two weeks of physics instruction that followed the PUM curriculum, only about 25% of students and student groups submitted responses of similar quality for the identification of assumptions and less than 10% of similar quality for the evaluation of the effects of assumptions (Fig. 1).If we combine proficient and partially proficient groups we will reach just above 50% on the identification of assumptions and less than 40% for evaluation of their effects.This is where feedback and revisions were important.Table 1 shows a group response on the first lab report that was scored "Partially Proficient."The students chose to organize their assumptions in a table in their report.While the students correctly identified a scientifically relevant assumption, they did not yet understand how to describe the effects of the assumptions and to validate factors they assumed to be true.After feedback that asked the students to clarify what it meant to describe the effects of an assumption and to clarify the word "accurate," the students asked questions and revised their work.In the revised report they clearly described the effect of the assumption on the prediction as the measured and traveled distances would not be equal if the car did not travel in a straight path.This process of revising their lab reports helped students grow as the year progressed.
Over the next five months, students continued working with assumption-associated scientific abilities.Students who did not understand what was expected of them in TABLE 1. Student group response for a sample assumption from Lab #1.The first row of the table contains their original submission; the second row is their revised explanation.Changes to the original response are in bold.

Assumption Effect Validation
The car will travel in a perfectly straight path.
The position (x) is accurate.
We gently tapped the car back into a straight path if it went to the side.The position (x) of the car is not affected by any deviations from the straight marked path, which would cause the measured distance travelled to be shorter than the actual distance traveled.
Because the car had a natural tendency to curve to the right, we gently tapped the car back onto the straight path if it looked like it was about to curve off to the side.
September began providing stronger support for assumptions they chose as relevant.The sixth formal lab report of the year asked students to design an experiment to test the impulse-momentum relationship.They had force and motion sensors, meter sticks, and stopwatches.One student submitted the following explanation for an assumption, "if the table/track was on an incline, then the change in velocity would have increased faster than if the table was flat which would mean that the force [of the string on the cart] would be lower than expected.This assumption was validated because when the cart was put on the track and no one touched it and there was no weight at the other end, the cart did not move, indicating that the table was flat."The student clearly described the effect of the assumption on the outcome of their experiment and the steps she took to validate it.
The students completed the mechanics portion of the curriculum in mid-February.Figure 2a illustrates their proficiency with identifying assumptions.After six months, 95% of students were able to demonstrate proficiency or partial proficiency with this ability.Figure 2b shows student proficiency with evaluating assumptions.In this same time period, 90% of students were able to demonstrate proficiency or partial proficiency with this ability.

V. DISCUSSION
The purpose of this case study was to find out how high school students develop assumption-associated abilities.Over a six-month period, high school students enrolled in a first year physics course worked collaboratively to design their own laboratory investigations.The students wrote up their findings in Google Documents and had opportunities to revise reports for repeated exposure to the scientific abilities.As illustrated in Fig. 2, 90-95% of high school students demonstrated proficiency or partial proficiency in assumption-associated abilities at the end of the mechanics portion of the curriculum.
The results of this study can be compared to findings at the college level.As stated earlier, prior research at the college level showed it took 5-8 weeks for 80% of students in the class to become proficient or partially proficient in a specific ability [12].Work in reference 12 was done at a large state university in an algebra-based course for science majors.More than 50% of the students in this course never had physics prior to college.With respect to identifying assumptions, high school students demonstrated 80% proficiency or partial by the sixth lab in January.With respect to evaluating assumptions, high school students were still below 60% after writing their sixth lab report.This is consistent with the college students who also took longer to demonstrate proficiency in this ability.However, between the sixth and eighth lab investigations, high school student proficiency in these scientific abilities increased to 90%; college students did not achieve this in the one semester their reports were studied.
This difference in proficiency between high school and college students in introductory physics courses might be the result of several factors.First, the high school students attended physics class every day having more time on task compared to college students attending labs once a week.Second, the high school students have the same instructor for all activities while college students have different instructors for lectures, recitations and labs.Third, college students wrote all reports individually, while high school students wrote some reports in groups.Lastly, high school students have more frequent opportunities to ask questions and revise their work.Revisions might provide additional exposure to the scientific abilities.

FIG. 1 .
FIG. 1. Student proficiency in assumption-associated abilities after the original submission of their first lab report both in groups (N = 12) and individually (N = 22).