A Longitudinal Exploration of Students’ Beliefs about Experimental Physics

The Michigan State University Physics & Astronomy Department has recently transformed its algebra-based, introductory physics laboratory curriculum. This transformed, two-course sequence, Design, Analysis, Tools, and Apprenticeship (DATA) Lab, emphasizes the development of experimental skills and laboratory practices and provides students with an authentic physics laboratory experience. Here, we will discuss the overall impact of the transformation on how students perceive experimental physics through the two course sequence: mechanics (Lab I) and electricity, magnetism and optics (Lab II). In both courses, data were collected pre-and post-instruction via the Colorado Learning Attitudes and Science Survey for Experimental Physics (E-CLASS); the results will be presented at the course-level and longitudinally. In both courses, the DATA Lab transformation had a positive impact on overall E-CLASS scores. Students in the traditional-to-traditional course sequence demonstrated an overall decline in their overall views about experimental physics. Students enrolled in the transformed-to-transformed course sequence showed an initial increase in their E-CLASS scores and they remained stable throughout the second half of the course sequence. Students in the traditional-to-transformed sequence experienced a signiﬁcant increase in their E-CLASS scores; however, it only occurred during the second half of the two-course sequence.


I. INTRODUCTION
Traditionally, physics laboratories have been designed in such a way in which the scientific concepts that are introduced in the lecture setting of a physics course are reinforced in the laboratory setting through a scientific experiment. However, the impact of the traditional laboratory style format has on student outcomes has been challenged by physics education researchers [1][2][3]. In 2014, the Physics Education Research (PER) community called for the design, implementation, and assessment of physics laboratories to focus less on supplementing physics content understanding and more so on the experimental laboratory practices that the students engage in while performing a scientific experiment. Kozminski et al. documented recommendations for the physics laboratory curriculum through the American Association of Physics Teachers' Recommendations for the Undergraduate Physics Laboratory Curriculum [4]. The recommendations emphasize the need for engaging students in practices such as experimental design, data analysis, and scientific communication in their physics laboratories.
In response to the national call from the PER community to focus on student engagement within the physics laboratory context, physics departments have been undergoing laboratory course transformations in the introductory, middledivision, and advanced laboratories. Some studies have investigated the impact of skill-based laboratories on student understanding of specific practices. For example, Pollard et al. explored student understanding of measurement uncertainty using the Physics Measurement Questionnaire (PMQ) [5,6]. In general, students that were enrolled in the introductory physics laboratory shifted their understanding of measurement uncertainty toward a more set-like reasoning rather than a point-like reasoning. Volkwyn et al. found that only 20% of a cohort of incoming freshman physics majors had a deep understanding of measurement uncertainty after the first introductory physics laboratory course [7].
Other researchers have accessed other laboratory practices outside of measurement uncertainty. Dunnett, Gorman, and Bartlett found that encouraging inquiry in the laboratory environment can have a strong impact on group work, querying, exploration and attitude [8]. Wilcox and Lewandowski investigated students' beliefs about the nature of experimental physics for first-year laboratory courses and beyond first-year laboratory courses [9]. In general, student views and beliefs about experimental physics decreased in laboratory courses that emphasized the reinforcement of physics content while their views and beliefs increased in laboratory courses that emphasized the learning of laboratory practices and research skills. In addition, Pollard and Lewandowski investigated the impact of a laboratory course transformation of a large introductory laboratory course on students' beliefs about experimental physics [10]. Although there was no differences in overall student beliefs post-instruction between the traditional and transformed courses, they showed that students did shift their beliefs with respect to the specific learning goals of the transformed course.
Recently, the Michigan State University (MSU) Physics & Astronomy department transformed its algebra-based, introductory physics laboratory curriculum for non-physical science majors [11]. In parallel with the research studies discussed above, this study will seek to explore the overall efficacy of the MSU laboratory transformation by answering the following research questions: What was the overall impact of the MSU laboratory course transformation on students' views and beliefs about experimental physics? and How do the student's views and beliefs about experimental physics change over the two-course laboratory sequence? Are these changes differentiated by how the students matriculated through the two stand-alone courses?
We recognize that Wilcox and Lewandowski "strongly recommend that instructors do not focus only on their students' overall E-CLASS score, as it does not represent students' performance around a single well-defined construct" [15]. The results presented below should be considered preliminary to a future study examining the individual E-CLASS statements which align with the overall learning goals of the laboratory course transformation.

A. Laboratory Course Transformation
The laboratory course transformation was implemented at MSU, a large, mid-western, land-grant university in the United States serving approximately 50,000 students. MSU has a Carnegie classification as "Very High Research Activity" (R1) [12]. In 2016, the undergraduate enrollment of MSU is 51% women, 77% White, 14% International, 8% Black/African-American, 6% Asian, 4% Hispanic/Latino, and fewer than 4% for other student populations [13]. For the undergraduate student population, ACT scores range from 23 to 29 (25th percentile to 75th percentile).
The newly transformed course, Design, Analysis, Tools and Apprenticeship (DATA Lab) was designed to focus on developing students' laboratory practices and research skills [11]. The two-course sequence consists of two stand-alone courses: mechanics (Lab I) and electricity, magnetism, and optics (Lab II). As most of the students enrolled in the twocourse sequence are non-physical science majors, the goal of the transformation is for students to develop research skills unique to a physics laboratory that can be utilized throughout the rest of their studies. Throughout the semesters, students explore classic physical systems as a tool to increase their overall laboratory skills (i.e., experimental process, data analysis, collaboration, scientific communication). Figure 1 illustrates the transformation over the time period studied. Overall, the course transformation took a total of four academic years with a gradual transition up to full-scale; the transition began with implementing the DATA Lab into approximately 10% of the laboratory sections and followed by approximately 50% of the laboratory sections. By the spring of 2018, all laboratory sections of the two-semester course sequence were fully transformed (see Funkhouser et al. for further details [11]).

B. Measures
The overall impact of the course transformation was measured by using the Colorado Learning Attitudes about Science Survey for Experimental Physics (E-CLASS) [14] throughout the course transformation. The E-CLASS is a validated research-based assessment tool developed to measure students' views and beliefs about the nature of experimental physics [15]. It consists of 30 five-point Likert scale items probing various aspects of experimental physics such as experimental design, uncertainty in measurement, and scientific communication. The scoring method of the E-CLASS is similar to the scoring method of the Colorado Learning Attitude about Science Survey (CLASS) [16]. Student responses are compressed into a three-point scale by combining "(dis)agree" and "strongly (dis)agree" into one category, respectively. The student's responses are then compared to an expert-like response and are coded with +1 if they align with the expert's response and a -1 if they are opposite to the expert's response; neutral responses are coded with a 0.
Throughout the two-course laboratory sequence, the E-CLASS was given as an online, extra credit survey pre-and post-instruction. Students received a 2% increase toward their in-class participation grade for completing both the E-CLASS pretest and the E-CLASS post-test. The student's in-class participation grade was worth 25% toward their overall grade.

C. Sample
The data analyzed in this study was collected in the algebra-based, introductory physics laboratory sequence, Lab I and Lab II, between the fall 2015 and spring 2019 semesters. As a control, student responses to the E-CLASS was collected in the traditional laboratory sections prior to the DATA Lab course transformation. For Lab I, the control semesters included fall 2015 to fall 2017 and for Lab II, data was collected in the traditional laboratories between the fall 2015 and fall 2016 semesters. Lab I Sample: For the entire period studied, a total of 2,661 students completed the E-CLASS pretest. Of those students, 1,714 students completed the E-CLASS posttest. Students that repeated the course were eliminated leaving a total sample of 1,473 students in Lab I that completed both the E-CLASS pre-and post-test (n trad = 1, 027 and n DL = 446). Lab II Sample: A total of 2,382 students completed the E-CLASS pretest with 1,300 students also completing the E-CLASS post-test. Like Lab I, students that repeated the course were eliminated. This left a total sample of 1,159 students in Lab II that completed both the E-CLASS pre-and post-test (n trad = 495 and n DL = 664).

III. RESULTS
To explore the differences in the student's overall views and beliefs about experimental physics between the traditional laboratory and the newly transformed DATA Lab, the students' overall raw E-CLASS scores were analyzed. Linear mixed effects (lme) modeling was used throughout this study to predict the overall E-CLASS scores over time. This modeling technique was used with students as a random effect to control for the within-subject correlation of the data [17]. The independent variables in our models were coded as the following: (1) time -the E-CLASS pretest was coded as 0 and the E-CLASS post-test was coded 1 and (2) type of course -traditional laboratory course was coded as 0 and the transformed DATA Lab course was coded as 1.
Over the time period studied, the course transformation in Lab I began at a later point in time than in Lab II (see Figure  1) which, in turn, resulted in an imbalance of sample sizes between the traditional laboratories and the transformed laboratories. Linear mixed effects models have the ability to control for the imbalanced sample sizes caused by this effect. Presented below are the results of the lme models disaggregated by course (Section III A) and longitudinally across the twocourse sequence (Section III B). Table I presents the student's average E-CLASS pretest and post-test scores; the results are presented as a percentage for each of the courses, as well as disaggregated by the type of course (traditional vs. DATA Lab). Overall, the E-CLASS scores of the students that were enrolled in the traditional laboratories decreased over a semester. However, in the transformed DATA Lab course, students' E-CLASS scores increased, on average, from pre-to post-instruction.

A. Course-Level Analysis
Within the transformed laboratories had an increase of 6%.

B. Longitudinal Analysis
The significant difference in the E-CLASS pretest scores between the Lab II traditional and transformed laboratories observed in the section above could possibly be influenced by how the students matriculated through the two-course sequence. For example, a student could have taken the traditional laboratory format for both courses, the transformed laboratory format for both courses, or a mixture of both. To explore the longitudinal results on how students perceive experimental physics throughout the two-course sequence, the Lab I and Lab II samples described above were combined. The total number of students with the four E-CLASS measures (pretest and post-test across both courses) was 454 students; a total of 189 students took Lab I and Lab II in the traditional style format (Trad-Trad), 141 students took Lab I in the traditional format and Lab II in the transformed DATA Lab format (Trad-DL), and 124 students took both courses in the transformed DATA Lab format (DL-DL). Figure 2 presents the student's average E-CLASS scores over the two-course sequence. In general, the overall E-CLASS scores for students enrolled in the Trad-Trad matriculation decrease over the two-course sequence. The overall E-CLASS scores for students that took both courses in the transformed laboratory format (DL-DL) increased initially with the Lab I course and then stabilized in Lab II. The students in the Trad-DL matriculation group demonstrated an initial decrease in E-CLASS scores in the traditional Lab I course but a large increase from pretest to post-test in the transformed Lab II course.
To explore this is further detail, a lme model was analyzed with the four E-CLASS measurements coded with respect to the time they were taken: Lab I pretest as 0, Lab I post-test as 1, Lab II pretest as 2, and Lab II post-test as 3. The stu- dents in the Trad-Trad group were coded as 0 , the students in the Trad-DL group were coded as 1, and the students in the DL-DL group were code as 2. The B and SE results are presented in percentages and are interpreted with respect to Lab I pretest (time=0) and students in the Trad-Trad category (group=0). Table II  However, the average E-CLASS scores between the Lab I post-test and the Lab II pretest was non-significant indicating that these students' views and beliefs about experimental physics remained stable between the two-course sequence. In addition, the average E-CLASS pretest scores with respect to the group membership was not significant. Comparing the trajectories from Lab I to Lab II, the student's E-CLASS shift was slightly more negative than the students in the Trad-Trad group [B = −4.99, SE = 2.5, t(1362) = 1.97, p < 0.05]. However for the DL-DL students, the shift from E-CLASS pretest to post-test in Lab I was positive [B = 9.72, SE = 2.6, t(1362) = 3.70, p < 0.001]. The difference between the Lab I pretest and the Lab II pretest was significantly negative for the Trad-DL group of students [B = −7.10, SE = 2.5, t(1362) = 2.81, p < 0.01] but was significantly positive for the students in the DL-DL group [B = 5.60, SE = 2.6, t(1362) = 2.13, p < 0.05]. The shifts in E-CLASS scores over this time period for the Trad-DL group was 7% lower than the shift for the Trad-Trad group of students. The shift in E-CLASS scores for the DL-DL group of students, however, was 5% higher than the shift for the Trad-Trad group. Lastly, over the entire time period of the two-course sequence (from Lab I pretest to Lab II post-test), the shift in E-CLASS scores compared to those for the students in the Trad-Trad group was 15% greater [B = 14.87, SE = 2.5, t(1362) = 5.88, p < 0.001] and 10% greater [B = 9.72, SE = 2.6, t(1362) = 3.70, p < 0.001] for the students in the Trad-DL group and the DL-DL group, respectively. Although the overall E-CLASS scores for the Trad-DL group decreased throughout the time period in which they were in the traditional laboratory format (Lab I), their E-CLASS scores by the end of the two-sequence course increased to a similar level of the E-CLASS scores for the students that took the transformed style format for both courses.

IV. DISCUSSION AND CONCLUSION
This study sought to explore the overall efficacy of the recent laboratory transformation at MSU. In both laboratory courses, overall E-CLASS scores from students in the traditional style laboratory courses decreased overall by 7% and 3% in Lab I and Lab II, respectively. However, in both of the stand-alone transformed laboratory courses that were developed to emphasize laboratory practices and research skills, the overall E-CLASS scores increased from pre-to postinstruction: 5% in Lab I and 6% in Lab II.
In addition to investigating the efficacy of the course transformation at the course-level, this study also explored students' views and beliefs about experimental physics over the entire two-course sequence. Students that were enrolled in the traditional style format for Lab I and Lab II demonstrated a continuous decline in E-CLASS scores from the beginning of Lab I to the end of Lab II. E-CLASS scores increased initially for the students that were enrolled in the transformed DATA Lab format for Lab I and Lab II and then remained stable for the rest of the course sequence. For students that were enrolled in the traditional laboratory for Lab I followed by the transformed laboratory for Lab II showed an initial decline in their beliefs about experimental physics; however, demonstrated a large increase in E-CLASS scores in the second-half of the course sequence.
Overall, the DATA Lab course transformation showed a positive impact on students' views and beliefs about the nature of experimental physics. As the E-CLASS was developed to probe multiple facets of the nature of experimental physics and Wilcox and Lewandowski suggest that researchers investigate student responses to the individual questions that are alligned with the overall learning goals of a course [15], in the future, we will further investigate the impact of the DATA Lab transformation on the item-level student responses.
This work was supported by the Howard Hughes Medical Institute, the MSU College of Natural Science and the MSU Department of Physics & Astronomy.