Developing student attitudes in the first-year physics laboratory

A goal of many university physics courses is that students increase their appreciation of physics as an evidence-based way of understanding the world, and that they begin to develop their self-identity as scientists. Discouragingly, studies have found that students attitudes about science are very difficult to affect positively. In our “structured quantitative inquiry” first-year physics lab course (SQILab) at the University of British Columbia, we have introduced weekly reflection activities specifically targeting students’ beliefs about the nature of science and their self-identification as scientists. We measure student attitudes using the Colorado Learning Attitudes about Science Survey for Experimental Physics (E-CLASS). Our preliminary results indi-cate that the SQILabs curriculum with these activities does not produce significantly better outcomes for student attitudes than SQILabs without these activities, but that student attitudes do increase in all SQILabs courses on the portion of the E-CLASS that targets central SQILabs learning goals.


I. INTRODUCTION
An important aim of many university physics and science courses is to develop students' attitudes and epistemologies: e.g., to build students' views of themselves as developing scientists and their understanding of what it means to learn and do science.Positive student attitudes, epistemologies, and self-identity as a scientist are linked with retention as physics majors and content learning [1][2][3]; understanding the nature of science is important for all scientifically literate citizens.
Surveys have been developed to measure student attitudes in lecture-based courses [4][5][6]; dishearteningly, studies using these surveys have found that student attitudes are very difficult to affect positively [4,6,7].Courses that have reported making a positive impact on student attitudes have typically included significant explicit components focused on developing student beliefs [7][8][9][10].
Recent work on student attitudes has targeted laboratory courses as potentially important environments for students to engage in authentic scientific practices and critical thinking [11,12].The Colorado Learning Attitudes about Science Survey for Experimental Physics (E-CLASS) is a tool designed to measure student attitudes particularly in laboratory courses [13]; it comprises 30 statements to which students must rate their level of agreement (see II C).In a study of several thousand university students, Wilcox et al. [12] found a small but significant decline in E-CLASS score (fraction of expert-like responses) over the duration of their physics courses.
A promising venue for improving student attitudes and epistemologies is the Structured Quantitative Inquiry labs (SQILabs), a first-year introductory physics lab curriculum [11].SQILabs focus specifically on developing students' understanding of the nature of scientific measurement and uncertainty.In each lab, students make quantitative comparisons between their measurements using uncertainties, then plan and iterate to improve their experimental and/or analysis methods.SQILabs students have been shown to carry the critical thinking practices they develop in SQILabs into their second-year physics lab course [11].
Holmes et al. [14] used the E-CLASS to compare student attitudes in SQILabs with those in a traditional lab course: attitudes in the traditional lab course became less expert-like, while those in the SQILabs remained constant.The results for the SQILabs students came as a surprise, since other measures (e.g., behaviors in the lab, written work in lab books) indicated that SQILabs students develop signficantly more expert-like epistemologies during the course [11].(In II D, we investigate the possibility that students do make gains on a subset of E-CLASS statements.)A possible explanation [14] is that although SQILabs students engage authentically in the scientific process, the SQILabs activities have addressed student attitudes and epistemologies only implicitly rather than explicitly: improving attitudes may require targeting in a way that is explicit to students [8,15].In this paper, we test how reflection activities may provide this explicit targeting: e.g., students reflect on the connection between their learning in physics class and the nature of science more broadly [9,14].
We have added reflection activities to two SQILabs-based courses offered at the University of British Columbia (UBC) during the 2015-2016 academic year, and administering the E-CLASS at the beginning and end of each course.We compare these courses with E-CLASS results from the traditional and SQILabs courses previously presented in [14], considering (1) all statements in the E-CLASS, and (2) the subset of statements most aligned with the SQILabs learning goals.We hypothesized that attitudes of students in SQILabs courses with reflection activities would improve during the course, and that they would improve more than those of students in SQILabs courses without reflection activities, which in turn would improve more than those of students in the traditional lab.We expected these improvements particularly on the E-CLASS statements that address the central learning goals of SQILabs, and perhaps also on the E-CLASS overall.
In II, we describe the courses, reflection questions, E-CLASS tool, and process of selecting statements to align with learning goals.In III, we present preliminary results from analysis of our E-CLASS data, which suggest that the ad- dition of reflection questions to the SQILabs curriculum did not have a significant impact on student attitudes, but which suggest also that the SQILabs curriculum did have a positive impact on the attitudes addressed by this selected subset of E-CLASS statements.In IV we discuss these results and describe directions for future work.

A. Courses
This work takes place in four introductory physics laboratory courses at UBC (see Table I).The traditional lab course (T1) is part of the most common path for life science majors.The other three courses are different years of the SQILabs curriculum (S1, S2R, S3R); these are the most common paths for physical sciences majors.Courses S1 and S2R are part of an enriched (honors) physics sequence, while S3R is not.(Both first-year sequences contain mechanics; the enriched sequence also contains introduction to topics like special relativity and quantum mechanics.)

B. Reflection questions
The curricula for all of the SQILabs-based courses (S1, S2R, S3R) are essentially the same, a main difference being the presence or absence of specific prompts that we call "reflection questions."In S2R and S3R, students were asked to answer weekly questions intended to help them view themselves as developing scientists and to connect their learning in the course to their everyday lives and to their future careers as scientists.In S2R, students answered the same two questions (written in their lab books) at the end of each lab session: "What have you learned in this lab that is useful for you as a scientist?"and "Where does this fit into the flowchart for how experimental physicists work?" (referring to a flowchart adapted from Zwickl et al. [16]).An example response to this question was, "...The activity [about histograms] really boosted our creativity and really made us thinking [sic] what is the most appropriate and accurate way to find the answer.It is similar like [sic] having a scientific question, then creating / designing an experiment to solve the question.This skill is very important as a scientist."Each lab session typically began with a short discussion of students' reflection responses from the previous week.
In course S3R, every week students answered different reflection questions appropriate to the point in the course.These questions were given as a weekly prelab assignment, answered through the university online platform.For example, in the week after students began practicing iteration to improve measurement uncertainty, students answered the question, "Think of a measurement you've made previously, outside this class (e.g., in another science lab course, or in your everyday life).(a) Describe the measurement very briefly.(b) What do you think is a major source of uncertainty in the measurement?(c) What are some reasons you might want to improve the quality of your measurement?(d) Brainstorm and write down some ways you could improve it."An example response to this question was: "I have measured my weight every day for a month.Once [sic] source of error was that I was not being consistent in the time of the day that I was weighing myself and as a result some of the daily fluctuations that I saw was as a result of taking measurements differently every day.In other words, the changes did not really reflect true changes in my weight...I should weigh myself every day at the same time during the day and should wear the same clothing every time I weigh myself.It is also important that I use the same scale every day." Another example of a reflection prompt used in course S3R is: "As you wrap up this course and look ahead, what are two things you have learned in [Physics XXX] that you plan to use in your future science courses and career?"In both S2R and S3R, students received participation credit for their responses to the reflection prompts.

C. E-CLASS tool
In order to measure the evolution of student attitudes, we gave students the E-CLASS (online) at the start and end of their course.(Students were offered a small amount of course credit for completing the survey.)For each E-CLASS statement, students use a Likert scale to answer the questions, "What do YOU think when doing experiments for class?" (personal beliefs) and "What would experimental physicists say about their research?"(beliefs about experts).The E-CLASS has been validated with interviews and a statistical study of thousands of students [12,13].
Following [12] and [17], we collapsed students' responses to E-CLASS statements from a 5-point Likert scale to 3point Likert scale (Strongly Agree and Agree are equivalent; Strongly Disagree and Disagree are equivalent), and then converted these to a 3-point scale (+1 for expert-like response, 0 for neutral, -1 for non-expert-like).In our analysis, we include only students who answered all questions on both the pre-and post-tests (and correctly answered a filter question to ensure that they were reading the questions).Numbers of students included in the study are listed in Table I.

D. Aligning E-CLASS statements with learning goals
Wilcox et al. [12] find that the E-CLASS does not contain groups of statements whose responses tend to vary strongly together; as such, they recommend that instructors focus attention on the subset of questions most aligned with their course's learning goals.Following this recommendation, prior to analyzing our students' E-CLASS responses, we selected the subset of statements that we felt were most aligned with the central learning goals of the SQILabs courses (including attitudinal goals of self-identity and understanding of the nature of science).We call these statements "Cen-tral+Reflection statements."Overarching learning goals of the SQILabs are that students will develop a functional understanding of measurement uncertainty and broadly applicable data analysis tools, make meaningful reflection on the quality of their experimental results, and understand and apply the iterative nature of science.
To start, each of us (LS, JI, NH, DB) classified the E-CLASS statements individually and found five statements that we all immediately agreed were central to the course.For example, "Calculating uncertainties usually helps me understand my results better" and "If I am communicating the results from an experiment, my main goal is to make conclusions based on my data using scientific reasoning" were found clearly to align with goals of SQILabs.We then discussed the remaining statements and reached agreement on twelve statements we all feel are central to the SQILabs learning goals: 2, 3, 5, 11, 13, 16, 17, 19, 22, 27, 29, 30.(See [12] for the list of E-CLASS statements.)This set includes several statements that we felt were emphasized by the weekly reflection activities: e.g., "If I try hard enough, I can succeed at doing physics experiments" and "Physics experiments contribute to the growth of scientific knowledge".We emphasize that we selected these statements before looking at E-CLASS data from S2R or S3R, and without explicit reference to E-CLASS results from S1 or T1.The left panel of Fig. 1 shows that the overall attitudes of students in the traditional lab fell while the overall attitudes of students in SQILabs remained constant.This confirms work by [14], who found similar results comparing only T1 and the first year of SQILabs (S1).This first year contained only students in the enriched (honors) course; our results show that these initial findings also hold for students who elected to take a non-enriched physics course (S3R).Interestingly, the pre-test scores of students in S3R are very similar to those in S1 and S2R (and different from those in T1)-despite potential differences in those students' initial attitudes about physics, evidenced by their different choices to take or not take enriched physics.(This similarity in S1, S2R, and S3R pre-test scores holds for the "Central+Reflection" statements also.)Although we had hoped that reflection activities might positively impact the overall attitudes of students in S2R and S3R, this does not appear to have happened.

III. RESULTS
The right panel of Fig. 1 shows that attitudes on the "Cen-tral+Reflection" statements became somewhat more positive for students in the SQILabs, and clearly fell for students in the traditional lab.Even on these selected statements, the reflection activities in the SQILabs do not appear to have produced an impact on student attitudes substantially different from that of the SQILabs without reflection activities (i.e., results for S2R and S3R are very similar to those for S1).
To understand better the relationship between the "Cen-tral+Reflection" statements and the rest, we examine the average difference between Pre and Post  have clear directions for analyzing data, I am not sure how to choose an appropriate analysis method" and "Calculating uncertainties usually helps me understand my results better."Of the 14 statements that showed positive shifts, half are "Central+Reflection" statements (slightly more than the overall fraction of these statements, 12/30 = 40%).However, the statement with the most negative shift is also a "Cen-tral+Reflection" question, "When doing an experiment, I just follow the instructions without thinking about their purpose."For any of the courses, despite our careful selection of the "Central+Reflection" statements, these do not seem to stand out as a clearly distinct subset.A factor analysis of student responses could be interesting in revealing whether distinct subsets of statements do exist, and whether these are different between the four courses.

IV. DISCUSSION & DIRECTIONS FOR FUTURE WORK
These preliminary results indicate that the attitudes of SQILabs students (measured by the E-CLASS statements aligned with the central SQILabs learning goals) did become more positive from the beginning to the end of the coursehowever, the difference was not dramatic, and was not noticeably larger for versions of the course with reflection activities (S2R and S3R) relative to the version without (S1).
A complication in interpreting these results is that students in both S1 and S2R comprise two distinct populations of students (both in the same lab course together): approximately half of S1 and S2R students are enrolled in a standard physics course with active learning techniques, while the other half are enrolled in a cohort-based interdisciplinary science program.Anecdotally, we believe that this latter group has more in-class experiences (outside the lab) that would enhance their attitudes about science than the former group.Teasing out the impact of these activities will require further study.
Despite this caveat, we consider preliminary explanations for our results.Reflection activities almost certainly need to satisfy specific criteria to be effective at stengthening positive student attitudes, and it is possible that the activities written for S2R and S3R did not satisfy these criteria; e.g., types of questions, timing and frequency, method of grading, type of in-class follow-up, etc., are all likely important.
Another possibility is that students are displaying a ceiling effect in their attitudes.Zwickl et al. [13] found that experts (faculty and researchers) held less than 90% agreement on six (20%) of the E-CLASS statements-i.e., even if students held the same beliefs as the population of experts, they might not reach an average score of 1.0.Perhaps a score of ∼0.6 (the approximate pre-test level of students in S1, S2R, and S3R) is close enough to the maximum score that producing a rise is especially difficult.
A third possibility is that there are aspects of student attitudes that the reflection activities helped to improve, but which the E-CLASS does not capture.For example, the E-CLASS does not explicitly measure to what extent students view themselves as scientists, or to what extent students connect what they learn in the physics lab to other science courses or their everyday lives-two areas of student thinking that we specifically aimed our reflection activities to strengthen.Other instruments [4][5][6] may be more appropriate for targeting these types of affective goals.
An important future avenue for investigating these issues will be analysis of students' written responses to the reflection prompts in courses S2R and S3R.We plan to develop coding schemes and rubrics to study the evolution of students' responses over the semester, to determine if there are changes in level of sophistication.We will also consider trying other tools to measure student attitudes in our courses, in particular aiming to measure students' self-identity as developing scientists and their understanding of the nature of science.

Figure 1
Figure1shows the average (3-point) E-CLASS score on questions about personal beliefs at pre-and post-test for each

FIG. 1 .
FIG. 1.Average (3-point) E-CLASS scores on questions about personal beliefs for the four courses (see Table I), at beginning (Pre) and end (Post) of the course.Uncertainty bars represent one standard uncertainty in the mean.Left panel shows results for all 30 E-CLASS statements; right panel shows results for the 12 "Central+Reflection" statements (see II D).
FIG. 1.Average (3-point) E-CLASS scores on questions about personal beliefs for the four courses (see Table I), at beginning (Pre) and end (Post) of the course.Uncertainty bars represent one standard uncertainty in the mean.Left panel shows results for all 30 E-CLASS statements; right panel shows results for the 12 "Central+Reflection" statements (see II D).

TABLE I .
Physics laboratory courses in this study.a Pre and Post E-CLASS scores averaged over all statements and all students; b Same as footnote (a) but only for "Central+Reflection" statements.Pre and Post E-CLASS scores and their uncertainties are depicted in Fig. 1.Pre a Post a Pre b Post b