Using custom interactive video prelab activities in a large introductory lab course

The large introductory physics lab course at the University of Colorado Boulder, which serves primarily engineering and physical science majors, was recently completely redesigned to align with new explicit learning goals. One of the learning goals of the new course was to have students enjoy working on physics experiments and to see value in experimental physics as a discipline. Additionally, we wanted to make the student workload consistent with a one credit course. To help achieve these goals, we created custom interactive videos that were viewed by the students before the lab to help them prepare for the lab activities. We present design principles for creating these videos, as well as data regarding student engagement and perceptions of this part of the course.


I. INTRODUCTION
Physics Laboratory courses are increasingly becoming the focus of investigations of student learning. Many groups are exploring how students can engage in scientific practices that are unique to the lab environment (e.g., designing and carrying out experiments, analyzing data, developing models of phenomena, and presenting results) [1][2][3]. There are also efforts to evaluate how effective labs are at preparing students to perform on conceptual exam questions [4]. All of these efforts are leading to significant transformations of lab courses at both the introductory [5,6] and upper-division level [7,8]. One common component of most of these lab courses is a prelab assignment to help prepare students for the lab activity and make the best use of the limited time students have working with the apparatus. Here, we present the design and implementation of custom-made, interactive prelab videos in a large introductory lab course. We present results of students' engagement with the activities and their perceptions of the prelabs gained through survey questions and focus groups.
There is significant research showing the benefit of students preparing for a lecture by reading a textbook or through web-based multimedia pre-lecture assignments [9,10]. These studies show the positive impact on students' performance on in-class conceptual questions and classroom discussions. A similar type of preparation for lab courses often includes a prelab activity, where students may read material, answer questions, or watch videos of lab procedures. Preclass activities are particularly important for lab courses, where students have a limited amount of time to work with the equipment. To make that time the most productive, students should have a basic understanding of the underlying theory, a broad idea of the goals for the lab, and some knowledge of the types of measurements and procedures they will need to complete. As a result, prelab assignments often need to be specially designed for a particular course. The impact and design of such custom prelabs have been studied extensively in the chemistry education community [11][12][13].
Recently, there was a large meta-analysis of research on prelab activities for chemistry labs [13]. Based on the metaanalysis, the authors argue there is significant benefit from well-designed activities, and they present five guidelines for developing prelab activities. Prelab activities should... ...be included in laboratory courses. ...be embedded into the overall laboratory learning process. ...focus on the whole task, drawing learners' attention to overall strategy and approaches. ...focus on supportive information. ...address the affective domain.
We believe the guidelines for chemistry labs likely apply to physics labs as well. Based on a set of guiding principles for creating prelabs and positive results from interactive video pre-lectures, we designed a set of online video prelabs with embedded questions for a completely transformed introductory lab course at the University of Colorado Boulder. This transformation included new learning goals, course structure, content, apparatus, student activities, and graduate teaching assistant training. Over two semesters of the new course, we collected data on student engagement with the prelabs, as well as responses to survey questions and feedback from student focus groups. These data help us to answer the following two research questions: To what extent do students engage with video prelabs? And do students report that the video prelabs help prepare them for the lab activities?

II. BACKGROUND
The transformed version of the introductory lab course focuses on five broad learning goals: three goals focused on content (the use of set-like reasoning when interpreting measurements [14], the ability to prepare publication-quality graphs, and an epistemology of experimental physics that aligns with the expert view) as well as two goals focused on student affect (positive attitudes about the course and about experimental physics as a discipline). Twelve lab assignments were backwards-designed from these learning goals. For a more complete description of the course structure see Ref. [15].
For each lab, the students work in pairs and have one hour and 50 minutes to complete their investigations and document their work in an electronic lab notebook. These notebooks, which must be completed during lab, are the primary deliverable, reflecting the fact that maintaining a proper lab notebook is a highly-valued practice among practicing researchers [16]. This limited work time places a premium on the students' preparation before the lab begins.
To compensate, video prelab assignments (with embedded multiple-choice questions) accompany each lab, drawing on general concepts from just-in-time teaching [17] and the socalled "flipped classroom" structure [18,19]. Such preclass videos can be more engaging than written materials [20], and are associated with positive affective outcomes for students [21]. In general, several studies of specific instructional video content document learning gains in introductory physics [9,10,22,23], and while the evidence from metaanalyses of general prelecture content is more mixed, there is at least clear evidence that students respond positively to its usage [19,21].
A meta-analysis by Agustian and Seerly [13] identifies five recommendations for the use of prelab activities, which we believe are also relevant here. The first is simply that "pre-laboratory activities should be included in laboratory courses"; our prelab activities are also generally in accordance with the remaining recommendations: • Pre-laboratory activities should be embedded into the overall laboratory learning process. Our prelabs are worth 12% of the students' overall grades, the second largest grade category behind the lab notebooks themselves (72%). We evaluate the success of this embedding below by looking at the fraction of students completing each assignment.
• Pre-laboratory activities should focus on the whole task, drawing learners' attention to overall strategy and approaches. Our prelabs always begin with a statement of the overall goal of the upcoming lab, and the structure of the video tracks the overall "arc" of the lab, with necessary skills and physics content presented in the context of the larger goal.
• Pre-laboratory activities should focus on supportive information. Agustian and Seerly distinguish "supportive" information, which is highly generalized and of high intrinsic complexity, from "procedural" information, which is specific to the steps involved in a particular lab activity. Correspondingly, our prelabs focus on the theoretical content that underlies the activity. In particular, while our prelab videos frequently show the equipment that will be present in lab, they rarely, if ever, present details about how specifically to use it. We note, however, that some student feedback has pushed back against this practice (see "Results" below).
• Pre-laboratory activities should address the affective domain. Following the literature [24,25], our prelabs adhere to both the "voice principle" (videos are narrated with a human voice) and "personalization principle," (videos should strike a conversational tone and be situated in in the same setting that students will later use). In particular, our prelabs are narrated by a faculty member (CGW) who has been a co-instructor of the course, and are filmed in the same lab space where the course takes place. We further attempt to address the affective domain by pointing out when the students will be using equipment, software, or analytic techniques that are employed by actual researchers in the field, rather than equipment primarily used in pedagogical settings.
In addition to these principles from Ref. [13], we designed our prelabs with three more guiding principles: • Pre-laboratory activities should be efficient and compact. Some evidence from Massively Open Online Courses (MOOC's) suggests that "shorter videos are much more engaging" [26], and while some authors caution against hard-and-fast rules, common guidelines advise videos of not more than six to ten minutes in length [26,27]. Eleven of our twelve prelab videos are shorter than seven minutes in length, with the longest running ten minutes and 33 seconds.
• Pre-laboratory activities should encourage engagement with the lab manual. Following evidence that prelecture assignments with quiz questions encourage students to read their textbook [28], we structure many (though not all) of the embedded questions to require students to consult their lab manual for information.
• Pre-laboratory activities should demystify any "black box" formulae. Though theoretical physics content is not a learning goal of our course, for affective and epistemological reasons, we intend that students should never feel they have been handed an inexplicable black-box formula to use in their lab. Consequently, when complex formulae are required, our prelabs engage students directly in the derivations of these formulae, often pausing to ask them a question about a key intermediate step before proceeding.
These are the principles that underpin the content of the videos. Regarding their development and deployment, we sought to strike a balance between practicality and production value. All videos were filmed with an entry-level DSLR camera using a basic lavalier microphone to record audio. In general, prelabs were loosely scripted, making note of sections where students would benefit from video of either specific equipment or a mathematical formula. Such sections were filmed first with voice-over audio and digitally generated equations. Remaining sections were filmed with a faculty member against a generic chalkboard background and edited with consumer software (Final Cut Pro X). Videos were then uploaded to a private YouTube channel with closedcaptions for accessibility, and incorporated into a third-party video quizzing service (PlayPosit [29]), which facilitates embedded questions (including rendered equations).

III. METHODS
To analyze the effectiveness of these prelab assignments, data were collected from three principle sources. First, the third-party video hosting service which was used for the course automatically recorded information on which students completed the assignments, how long they took to complete the assignment, and which questions they answered correctly (in addition to other information not considered here). We examine data for the first two semesters of the course after it settled into its ongoing format (the very first semester of the transformed course involved slightly different prelab structure and content) and, except for completion rates, reject any datapoints from students who did not attempt the prelab in a given week. The result is approximately N = 1200 data points per prelab across the two semesters.
For each of the aforementioned quantities, we examine the mean, standard deviation, and standard error. For the assignment completion time, we also consider the median value, since for each prelab it was common for a few students to log unusually long completion times (for example, a time greater then 24 hours). Since such data points seem unlikely to represent the student's actual expended effort rather than a logistical anomaly (e.g. an assignment window left logged in overnight), an outlier-resistant measure of central tendency gives a more accurate picture in this case.
All students were also given the opportunity to complete an online survey at the end of the course, which included ques-tions about measurement uncertainty, as well as questions directly related to the course structure and content. One of these questions asked students to respond to the prompt "How well did the prelab videos prepare you to do the corresponding labs?" on a Likert scale from "very well" to "very poorly." Finally, during one of the early semesters of the transformed course, five small focus groups (3-5 students each) were run to discuss various aspects of the course, including the prelabs. Participation was solicited via email to all enrolled students a couple weeks before the end of the semester. BP, who was not an instructor of the course or known to the students in any other official capacity, was the email contact and facilitated the focus groups themselves. Groups were formed to prioritize grouping together people of a single gender and major area of study [30], within scheduling constraints. The students were assured that instructors of the class would not have access to the data from the focus groups until after their grades were submitted. Each participant was financially compensated for their participation. The focus group itself was run as a semi-structured interview with questions predetermined by BP and HJL. BP took written notes during the interviews, and they were video and audio recorded. Recordings of particular sections of interest were transcribed for further analysis. For this particular work, representative quotations were selected from among all statements made that specifically mentioned the prelab videos.
It is important for context to understand the general demographics of the students in the course; although billed as a freshman-level course in experimental physics, the students are primarily a mixture of physical science and engineering students, with a nontrivial fraction (33%) who have been at CU for four or more semesters. The demographics of the students in the class as reported by survey include 28% women, 71% men, 1% non-gender conforming.

IV. RESULTS AND DISCUSSION
We begin by presenting the analytics extracted directly from the PlayPosit system, which are summarized in table I. These data directly address the question of student engagement and also provide important context for the question of how well the students felt they were prepared by the prelabs. We note that the fraction of students completing the prelabs is consistently high and does not significantly decline throughout the semester, which one might expect if students did not find them worthwhile.
The average scores on the prelab assignments are generally strong, notable because many of the prelab questions cannot be answered with pure background physics knowledge, but require engagement with the lab guides and course materials. Indeed, the distribution of prelab scores is skewed significantly upward, as seen in Fig. 1. We initially sought to examine the efficacy of the prelabs by testing for correlations between a student's prelab score and their score on the associated lab, but unfortunately these high-end saturation effects TABLE I. Analytical information from student prelab video logs (N ranging from 1156 to 1236). "Completion" is the number of students who fully completed the prelab as a fraction of the final course enrollment. "Avg score" is the mean score among all students who started the assignment; uncertainty is given by standard error. "Median time" is the median elapsed time from when a student opened a video to when they submitted it for a grade. (which are present in lab scores as well) make such correlations of limited value. The median elapsed time spent on each prelab suggests that, in general, most students are not spending more than 15 minutes on the assignment. Except for the first and last weeks, which are intentionally shorter and easier, they also suggest that a typical student spends roughly a minute to a minute and a half per question (calculated from the median elapsed time minus the length of the video itself, divided by the number of questions). These numbers are consistent with the idea that students do not find the prelabs overly burdensome.
We note also that we have found very few instances (less than 1% of cases) where a student's total elapsed time is nearly the same as the base video run time. Such an occurrence might represent a student either randomly clicking through the questions or blindly copying answers from classmates; the apparent paucity of such behaviors is also consistent with students who do not find the prelabs too onerous and who might in fact find value in completing them.
Against this backdrop, we can assess the students opinions more directly from their responses to our anonymous survey questions and from their comments in focus groups. In response to a Likert-style survey question asking "How well did the prelab videos prepare you to do the corresponding labs?" 80% responded either "well" (50%) or "very well" (30%), with uncertainties less than ±3% as estimated using the binomial proportion confidence interval at the 95% confidence level. Only 6.6 ± 1.4% percent believed they were "poorly" or "very poorly" prepared.
This sentiment also appeared as a common theme in the focus groups. In one, a prompt from the facilitator ("One other aspect about 1140 were the prelab videos. How well We do note, however, that in two of the interview groups, students said they felt the utility of the prelabs varied throughout the semester, identifying some prelabs as more useful and applicable than others, though not all students in the groups expressed agreement.
Other comments about the prelabs that were present across focus groups expressed appreciation that they were "short," and "not...boring." Multiple students also commented on the advantages of video prelabs over written prelab materials, saying for example that "you can go at your own pace" and that they were "more engaging" than written assignments.
One of the more negative themes that emerged was that many students wished the prelabs had showed more details of the actual lab equipment being used, instead of emphasizing background and theory. As noted above, however, this was in fact a conscious design choice to allow the prelabs to focus on supportive rather than procedural information [13].

V. CONCLUSIONS AND FUTURE RESEARCH
We have described the creation of a set of interactive "prelab" videos for use in a transformed introductory physics lab course. The design principles of these labs are consistent with the recommendations from a recent meta-analysis focusing on prelab videos in chemistry courses [13]: the prelabs are embedded in the course structure, they describe the overall lab strategy, they focus on supportive information, and they address the affective domain. We also highlight three additional principles that guided our design: the prelabs are compact, they encourage engagement with the lab manuals, and they attempt to demystify "black box" formulae.
We asked two research questions: First, to what extent do the students engage with the prelabs? And second, do the students report that the prelabs prepared them well for lab? In answer to the first question, our preliminary results show that very high percentages ( 90%) of the students are completing the prelabs, with the vast majority of students spending a nontrivial amount of time engaging with each question and an average time per question of well over a minute.
In answer to the second question, survey results suggest that a large percentage of the class (roughly 80%) do feel that the prelabs prepared them well. This was also expressed in focus groups, although some students also stated that certain prelabs were more successful in this respect than others.
While students seem to generally believe that the prelabs prepared them well, in future research, we would like to know whether there is evidence to show that this belief is correct. Saturation effects prevent us from drawing meaningful conclusions about correlations between prelab scores and lab scores. But more targeted data focusing on performance on individual prelab questions and specific lab components might be illustrative. It would also be interesting to look at correlations between lab performance and time spent engaging with the prelabs. While student background knowledge would be a confounding factor here, this might be minimized by looking only at prelab questions that engaged with specific lab details rather than general physics theory.
Finally, it would be useful to examine in greater depth whether the prelab design principles that we have drawn from the world of chemistry truly translate to the physics community, and which, if any, discipline-specific principles might be added in the physics context. These and other questions could be addressed with more targeted data collection during future semesters of the transformed course.