The impact of metacognitive activities on student attitudes towards experimental physics

For the past three years, I have used the Colorado Learning Attitudes about Science Survey for Experimental Physics (E-CLASS) to monitor how an advanced lab course with a significant studentdriven project impacts student attitudes about experimental work. During that time period, I have increased the use of metacognitive activities that ask students to reflect on their approaches to making decisions and handling problems they encounter in the lab. Here I report on the correlation between the introduction of metacognitive activities and changes in some responses on the E-CLASS survey, as well as providing a qualitative overview of the students' metacognitive reflections.


I. INTRODUCTION
The curricular laboratory is an essential part of the undergraduate physics curriculum.An increasing emphasis has been placed on providing students the opportunity for open-ended, inquiry-based laboratory experiences [1,2] as a way to increase student interest and engagement.The AAPT Recommendations for the Undergraduate Physics Laboratory Curriculum [3] suggest that students have the opportunity to see a project through from conception to completion.While open-ended projects can be engaging, there are also challenges, particularly if students' previous lab experiences have been in traditional curricular labs that provide a clear outline of what to do.Students find it difficult to transfer skills from structured problem solving to ill-defined problems, but introducing metacognitive support activities has been shown to enhance open-ended problem solving [4,5].
Metacognition refers to thinking about one's own thinking.Models of metacognition often include two main components -knowledge of cognition (knowing how and why to do things) and regulation of cognition (planning, monitoring, and evaluating strategies for learning or problem-solving) [6,7].The inquiry laboratory where students must analyze the problem or question at hand, select an approach to engaging with the laboratory activity, reflect on how things are going as the lab proceeds, and make adjustments accordingly provides excellent opportunities for students to practice the regulation of cognition [7].Students may not engage in metacognition without explicit instruction, but studies have shown that introducing metacognitive activities in the classroom and the laboratory helps students develop the appropriate learning strategies and problem solving approaches [8][9][10][11][12].Encouraging students to engage in metacognition in the laboratory not only has the potential to deepen learning, it also has the potential to impact the affective domain, reducing frustration and building confidence.
This paper describes how the Colorado Learning Attitudes about Science Survey for Experimental Physics (E-CLASS) [13,14] and the introduction of metacognitive activities were used to reform an advanced lab course at one small liberal arts college.We report on E-CLASS results from spring 2013 (course enrollment: 19 students; matched pre-and post-E-CLASS surveys: 10 students) and spring 2015 (course enrollment: 22 students; matched preand post-E-CLASS surveys: 20 students).Although the number of students is too small for meaningful statistical analysis, the E-CLASS results suggest that the introduction of metacognitive activities in an advanced lab where the laboratory work is not carefully scripted may improve students' enthusiasm for experimental work and confidence in their ability to be successful in such work.

II. COURSE CONTEXT
Physics 342 Contemporary Experimental Physics is a ten-week advanced laboratory class, typically taken by physics majors in the junior year.The course involves three class meetings per week, and one four-hour lab period each week.However, students are expected to continue and complete laboratory work outside of the scheduled laboratory hours.
In 2012, the course was redesigned provide more openended laboratory work.Students complete three two-week labs (from five possible choices), designed by the instructor, which are significantly less scripted than what students have seen in intermediate-level lab courses.In the second part of the course, students design and carry out an experimental project of their choosing.Projects are carried out in groups of two or three, and planning begins in the second week of the course.
The three stated learning goals for the course are: 1) That students will be able to estimate the quality of the measured data and compare measured data with predictions from a model or from previous work; 2) That students will be able to build or assemble a measurement system from a given set of components, test and revise the measurement system if it does not work as expected, and keep a clear record of their work as they do this; and 3) That students will be able to clearly and convincingly communicate experimental work and results to an appropriate audience, either orally or in writing.Particular skills or experiments are not a focus of this course.Rather, the goal is for students to gain an appreciation for the process of doing experimental work.Although not stated explicitly in the syllabus, instructors aim to give students a positive experience in the laboratory that builds their confidence to handle future laboratory situations that they might encounter.
Since 2013, the E-CLASS survey has been employed to evaluate the course.The survey assesses students' views about their strategies, habits of mind, and attitudes when doing experiments.The E-CLASS survey uses a likert-type scale, asking students their opinions about particular aspects of experimental work, and then compares how expert-like student attitudes are pre-and post-course.

III. METACOGNITIVE ACTIVITIES
Many students come to this course without previous curricular experience that includes open-ended laboratory projects, and the transition produces frustration that manifests itself during lab time and in the course evaluations.Metacognitive support mechanisms can help students move from structured to open-ended problem solving [4,5], and after 2013, metacognitive support activities were introduced in the course.Two approaches are available to support metacognition, strategy training and creating social support [15]; both approaches were employed.
Strategy training ranges from teaching students strategies for domain-specific problem-solving tasks to prompting reflections that help students generate questions about and monitor their metacognition.We used a series of prompts designed to elicit narrative statements about what the students did as well as asking the students to make action statements about what they planned to do differently going forward.Students were asked to respond to prompts at the end of the first two instructor-designed labs in the first half of the course (week 2 and week 4).Students provided individual written responses to five prompts.The five prompts were:  Tell me a bit about how you approached the lab. When you ran into problems, what was the strategy your group employed for troubleshooting the problems you encountered?
 What types of pre-reading or additional research did you do with regards to your particular lab?  When you asked for help, who did you seek help from and what kinds of questions did you ask?  What is one thing that you would do differently when tackling labs going forward?
In addition to the individual reflections in the first half of the course, class time was set aside during the middle of the open-ended final project (week 8) for a reflective conversation among all lab groups.This activity was designed to develop a supportive social environment for metacognition.Social modeling of metacognition allows peers to observe how others engage in problem-solving and to learn from and build on the approaches that others take [15].Each lab group was asked to come prepared to discuss two prompts:  What is a problem that your group encountered when working on your project in the past week, and what was your approach in trying to address it? What are your project goals for the coming week and how do you plan to pursue those goals?
Through these in-class conversations, students shared their challenges, concretely presented their plans, and received feedback from other groups.In this manner, students were able to hear each other's approaches to troubleshooting, planning, and managing set-backs.
In addition to the introduction of the individual reflective prompts and the reflective group discussions, the revised course also included full class discussions that explored methods for addressing difficulties that arise in the laboratory, personal reflections by faculty members and students who had participated in research experiences, and historical case studies.Although the activities were modest, the explicit discussion of experiences with and expectations for experimental work was intended to normalize the feelings that students had when progress was slow or frustration grew.

A. E-CLASS results
Figure 1 shows the overall E-CLASS results before the introduction of metacognitive activities (2013) and after the full implementation of metacognitive activities (2015).In both courses, students show an overall increase in expertlike responses after taking the course.However, several items on the E-CLASS survey show surprising differences.
We identified three E-CLASS items that seemed to highlight student enjoyment of and confidence in carrying out experimental work.The item that initially caught our attention, and led to the introduction of metacognitive activities, was, "I don't enjoy doing physics experiments."In 2013, the class showed a decline in expert-like responses on this item, indicating students enjoyed doing experimental work less after taking the course, despite showing an overall gain on the E-CLASS survey.In 2015, after revising the course and introducing metacognitive activities, the class showed an increase in the percentage of students giving expert-like responses (from 70% to 85%) suggesting that the course contributed to increased enjoyment of experimental work.
There were two additional E-CLASS items related to student confidence where there were gains in expert-like responses after the introduction of the metacognitive activities.Students had more expert-like responses to the items, "If I wanted to, I think I could be good at doing research" and "When I approach a new piece of lab equipment, I feel confident I can learn how to use it well enough."Sudents in 2013 were relatively confident in their skills at the beginning, and there was no change in that response after the course.In 2015, students began with less confidence, but showed gains in expert-like responses to questions of confidence in doing research (from 80% to 95%) and confidence in using new equipment (from 50% to 70%).

B. Additional considerations
One limitation in comparing the results from 2013 and 2015 is the different response rates.In 2013, slightly more than half the class took the E-CLASS survey, while in 2015, 22 of 24 students took the E-CLASS survey.The self-selected students who took the survey in 2013 may have been more confident in their lab skills.
A second limitation of the data is the inability to connect E-CLASS responses with previous coursework.The curriculum includes two other elective upper-level lab courses (electronics and optics) which may be taken before taking Phys 342.The fraction of students in any given year who have taken other advanced lab courses may impact the E-CLASS pre-test responses.Another factor to consider when exploring the difference in the E-CLASS results between the 2013 and 2015 courses may be the mindset of the students.Much attention has been given to the growth mindset and the impact it can have on student learning and success [16].Less than half of the students in 2013 agreed with the E-CLASS item, "Nearly all students are capable of doing a physics experiment if they work at it."The students correspondingly showed no change in their confidence in doing experimental work after taking the course (although confidence was already high).On the other hand, in 2015, 65% of the students agreed or strongly agreed that if students worked hard they could be capable of doing a physics experiment.The 2015 E-CLASS respondents showed an increased confidence in their ability to do research and approach new equipment.Whether this is attributable to the introduction of metacognitive activities in the class or the result of other factors contributing the mindset of students, the results suggest that further research be done on the potential benefits of introducing metacognitive prompts into an advanced lab course.

C. Responses to metacognitive prompts
Detailed examination of the responses to the metacognitive prompts is an area of future work.Most responses are narrative descriptions of what was done in the lab.Particularly interesting are the responses to the prompt asking students, "What is one thing you would do differently when tackling labs going forward?"The responses generally fell into two main categories: 1) group dynamics (time management and sharing work equitably among group members) or 2) pre-lab work (either thinking more about the models, instruments, or approaches in advance or spending less time getting bogged down in nitty-gritty details in advance).In addition, several students responded to this prompt without any concrete suggestions of how to do things differently, but rather with a discussion of their frustrations.One student wrote, "I know I will try to focus on how certain problems and issues during the lab are precisely the point of the experience (although I will still allow myself to get frustrated at the unhelpful ones)."Comments like this one suggest that the introduction of discussions during class time, both about the nature of experimental work and the importance of struggling with challenges independently as preparation for the open-ended project work, normalized the frustration that the students felt.

V. CONCLUSIONS
Revising an advanced lab course to include to include metacognitive support activities, including individual reflective prompts and in-class discussions that encourage social modeling of metacognition, has the potential to improve outcomes in the affective domain.E-CLASS results suggest that these modest revisions may improve students' enthusiasm for experimental work and confidence in their ability to be successful in such work.To better understand exactly what role the metacognitive activities play, future work will need to analyze students' written reflections in detail and explore how they explicitly connect with student attitudes captured by the E-CLASS survey.

FIG 1 .
FIG 1. Overall E-CLASS score on "What do YOU think…" statements for Carleton College (patterned gray) and for all courses administering the E-CLASS survey (dark gray).Students showed more expert-like responses in their attitudes about experimental physics after the advanced lab course, both before (2013) and after (2015) the introduction of metacognitive activities.

TABLE I .
Percent of expert-like responses for two offerings of the advanced lab course, one of which included metacognitive (MC) activities and one of which did not