Interactive Video Enhanced Tutorials: A Tool for Promoting Self-Paced Learning and Engagement for Future Engineers

— Learning in the present times has required not only advanced technology but the use of technology that aids in promoting learning that is both engaging and self-paced with a balanced mix of tempo and experience. This Research to Practice Work-in-Progress study involves interactive video enhanced tutorials (IVETs) that were developed as part of an ongoing NSF funded project to promote problem-solving in physics. IVETs are short, web-based activities taking students 10-15 minutes to complete. In this WIP paper, the researchers focus on a cross-section of the work that is part of the ongoing study and report impact of teaching problem-solving in the context of three groups namely, IVET group, video-only group, and no treatment group. The students are all engineering majors enrolled in a first-semester calculus-based physics course. Impact was also analyzed in relation to student demographics, with subgroups including women and under-represented minorities. This research study employs quantitative (demographic, follow-up problem) and qualitative data (student artifacts), along with mixed methods analysis to report the emerging results.


INTRODUCTION
One of the challenges of teaching, particularly in large enrollment courses, is being able to offer personalized instruction that effectively meets the diverse needs of students. Supplemental online instruction delivered outside of class shows promise in this area. This work-in-progress paper reports on web-based Interactive Video-Enhanced Tutorials (IVETs), which are designed to help students learn expert-like problem-solving approaches for core physics concepts or principles, such as Newton's Second Law or conservation of energy. Each IVET focuses on a single physics problem and guides students through the problem-solving process via a series of multiple choice questions interspersed with videos of a narrator, who serves in the role of a tutor. Students must answer each question correctly before proceeding in the IVET. Feedback is provided for both incorrect and correct answers, and the reasoning behind each step in the problemsolving process is emphasized as part of the feedback. The IVETs are assigned as homework and each typically takes 10-15 minutes to complete. Throughout the IVET, students can choose to receive guidance in either video or text form, and they may switch at any time. This allows students who are proficient in the material to choose text and navigate the IVET more quickly, while students needing additional support can benefit from the more detailed video feedback and summaries. This paper presents a study around one of the IVETs and its impact on student problem-solving abilities.

II. MOTIVATION AND BACKGROUND
Problem solving is an important learning outcome for introductory physics courses. Unfortunately, research has demonstrated that even students who understand the related principles and concepts often struggle when applying this knowledge to solve problems [1]. In these cases, students could benefit from deliberate practice [2], which goes beyond repetition and involves activities designed to explicitly improve performance through the use of multi-step problems that students are likely to find challenging, scaffolded support, and targeted feedback [3]. In addition, deliberate practice involves providing students with many opportunities across different types of problems to practice applying expert-like problem solving strategies, with particular emphasis on the reasoning behind how decisions are made for using certain principles and procedures [4].
The design of the IVETs builds on the work of DeVore and Singh [5,6], in which they created interactive problemsolving tutorials that provide students with deliberate practice for some of the more challenging but common problems assigned in introductory physics courses. Their tutorials are launched through PowerPoint and they found that the mode of delivery did not engage students with the tutorial to the extent intended. That is, they found that when students worked on various tutorials at home over the course of a semester, mean scores for follow-up quiz problems completed in class were in the 45%-55% range. Unfortunately, these scores were only 5%-10% higher than the mean scores of students who did not complete the tutorials. On the other hand, when students completed the tutorials in a one-on-one setting under the watch of a researcher who ensured that students were using them as intended (i.e., thinking through each question and paying attention to the feedback), the mean scores for the follow-up problems ranged from 83%-96%. The much lower performance of students who completed these tutorials at home suggests that when completing the tutorials on their own, students often do not engage with the tutorials deeply, thus benefiting much less from these self-paced learning tools than they would otherwise.
Under prior NSF funding, we have developed a web-based application for use with Interactive Video Vignettes, which focus on building students' conceptual understanding through interactive activities in which they answer questions and receive individualized feedback. Research has shown the IVVs to be effective for engaging students in the learning process and addressing common misconceptions [7,8].
Given our prior success in engaging students with online, interactive, self-paced learning tools and the potential to use these types of tools to help students learn problem solving, under more recent NSF funding, we have developed a new set of tutorials, some based on those of DeVore et al. and some that are new. Rather than PowerPoint, the IVETs are delivered through an expanded version of the web-based application used in the IVV project, and the IVETs use a similar design in which students are guided via the use of multiple choice questions and individualized feedback, with the explicit goal to help them learn effective problem solving strategies for core concepts/principles of physics.

III. EXAMPLE: TORQUE AND ROTATION IVET
The IVET around which the study in this paper was designed involves a typical Atwood machine problem that includes two masses hanging from opposite sides of a pulley, with each attached at a different radius. Values are provided for the two masses, radii, and moment of inertia of the pulley. The problem asks for the angular acceleration of the pulley and the tensions in the two strings. This problem involves integrating multiple concepts (torque and Newton's 2 nd law) which is quite challenging for students, thus providing an appropriate context around which to design an IVET. The IVET begins by presenting the problem to be solved, and students are subsequently led through the problem solving process via a series of multiple choice questions that were developed to scaffold one another and guide students through key steps that experts would use in approaching this problem. This includes identifying the physics principles needed to solve the problem (see Fig. 1), drawing free body diagrams for each mass along with an extended free body diagram for the pulley, applying Newton's Second Law for both linear and rotational motion, and ultimately solving after setting up three equations with three unknowns. Students can return to the problem statement or switch between text and video at any time using the buttons at the bottom of their screen (see Fig.  1). For both incorrect and correct answers, feedback is provided and includes the reasoning behind the use of each step in the problem. Given the complexity of this problem, students can opt to watch a summary of the solution completed so far at various points along the way, including a full 4.5 minute video summary at the end that carefully steps students through the entire problem-solving process (Fig. 2).
An important part of the IVET design that was not mentioned previously is that each involves the application of multimedia principles of learning [9] that are grounded in research in human learning and memory [10]. That is, students have control over their journey through the IVET, which research has shown to better motivate students to engage resulting in improved learning [11]. For example, within the IVET, students can choose their own pace, mode of presentation (text or video), and opt to watch various video summaries. The IVETs also include support to encourage students along the way while providing the reasoning behind each step taken, similar to what a personal tutor would do [12].

IV. RESEARCH QUESTIONS
This work in progress paper presents a study around two research questions (RQ), including: 1. How does performance on a follow-up problem differ for various treatment groups to understand IVET's impact on student problem-solving abilities?  2. How does performance on a follow-up problem differ for various demographic groups to understand IVET's impact on student problem-solving abilities?

A. Population
This study involves engineering majors enrolled in one of two sections of a first-semester calculus-based introductory physics course at a Midwestern University in the USA. The two sections met for 50 minutes, three days a week, at either 9 AM or 10 AM. They were taught by the same instructor and involved the same instruction and assignments. However, for the purposes of this study, one section was assigned as homework the "Torque and Rotation" IVET, while the other section was asked to watch the non-interactive video summary of the problem solution found at the end of the IVET (Fig. 2). Although all students were assigned to either complete the IVET or to watch the video solution, not all students completed the assignment, and participation was tracked through log files created when students opened the online application. This provided three groups for comparison, including the IVET group (interactive), video-only group (non-interactive), and no treatment group.

B. Procedure
All students, regardless of group, were given a related follow-up problem to complete as a quiz at the beginning of the next class following the assignment (Fig. 3). The followup problem was designed to assess key learning outcomes from the IVET, including students' ability to draw the necessary free-body diagrams, apply Newton's Laws for both linear and rotational motion, and set up multiple equations with multiple unknowns. Although the general procedure is similar to that of the IVET, the surface features of the problem are different and research has shown that students often answer such questions differently as a result [13]. Students' written solutions were scored based on correctness as well as coded for emerging themes by the course instructor and an instructor of a different course, both of whom are physics education researchers and involved in the development of the IVETs. The maximum that a student could earn on the followup problem was 8 points, where 2 points were provided for a correct free-body diagram of the box, 1 point for recognizing that the tension in the string is not equal to the weight of the box, 1 point for applying = , 1 point each for applying Newton's Second Law to linear motion and rotation, and 2 points for a final correct calculation (solving two equations with 2 unknowns) and unit. Scores were separately tallied and compared for the three groups of students in the study. In addition, all students' written solutions were coded based on emerging themes, including practices we expected to see given the common learning outcomes targeted in the IVET and video summary. This was done to provide more specific detail about how the different treatments impacted students' problem-solving practices for this physics topic (see Table I).  (7) Hispanic/Latino, and the rest as others (multiracial and unknown). The control (no treatment) groups for both lecture sections were combined as there were no significant differences on the follow-up problem outcomes between them. The final sample size and follow-up problem averages out of 8 points are shown in Table II for each of the groups.
In order to determine whether or not the groups were similar, ACT Math scores were compared, and no significant differences were found between the three treatment groups. ACT Math scores were used here as these scores have historically shown moderate correlation to students' exam performance, which essentially measure student's ability to solve various physics problems.
After testing for assumptions, a one-way ANOVA yielded a statistically significant difference in the follow-up problem scores for all three treatment groups (F = 11.80, df = 2, 197, p < 0.05) with a marginal large effect (η 2 = .11; suggesting 11% of the variance in the follow-up problem due to the type of treatment [14]) and an observed power of 0.99. Post hoc analyses, where Scheffe was used due to unequal group sizes [15], revealed that the IVET group performed significantly better (p < 0.001) than the control (no treatment) group, and significantly better than the video-only group (p = 0.013). According to Leven's test, the homogeneity of variance assumption was satisfied (F (2,197) = 1.29, p = 0.278). Normality was a reasonable assumption for both groups on the follow-up problem scores as tested by skewness and kurtosis values and Q-Q plot [14]. Since students were not randomly assigned, the independence assumption cannot be assumed.  Table III shows the follow-up problem performance for the different demographic groups. Because of the small sample sizes, averages on the follow-up problem are included but no statistical analyses were conducted. Additional data will be collected in the future to complete this part of the study. The themes that emerged from coding students' handwritten solutions to the follow-up problem were tallied by treatment group. As shown in Table IV, although the percent of students that were able to correctly solve the problem is low regardless of treatment group, those who completed the IVET or watched the video solution were more likely to approach the problem with a plan. That is, they were more likely to apply the problem solving approaches targeted in the IVET and video compared to the students who received no treatment, where just under half attempted to solve the problem by haphazardly applying equations based on surface features of the problem. In addition, those who completed the IVET, when compared to the video-only group, were more likely to approach the problem with a clear plan, and many more applied physics principles and recognized that the tension is not equal to the weight of the box instead of just assuming it is. The IVET students were also more successful in drawing a correct free-body diagram for the box, which was necessary for solving the problem by setting up two equations with two unknowns. These findings demonstrate the importance of providing students with experiences that involve deliberate practice, guidance, and feedback, as well as that mentally engage them in the learning process in order to help them learn approaches to problem solving. In answer to the first RQ, the data indicate that the Torque and Rotation IVET positively impacted students' problemsolving abilities because students who completed the IVET outperformed students provided with only a video solution, while both outperformed the group that received no treatment on the follow-up problem. In addition, the students in the IVET group were more likely than students in the video-only group to apply expert-like strategies, such as drawing a correct free-body diagram, recognizing that the acceleration of the block was non-zero, and applying Newton's 2 nd Law separately to the box and pulley to set up 2 equations with 2 unknowns. Although the video treatment students significantly out-performed those who received no treatment, with a medium effect size of 0.53, the study outcomes demonstrate the limited impact of videos, which can easily become passive learning experiences that are less able to motivate or engage students. This is important given the increased use of video lectures, such as Khan Academy, to supplement in-person instruction, particularly in the flipped classroom setting. In addition, unlike DeVore and Singh, who found that the tutorials needed to be completed in person to be effective when delivered through PowerPoint, we have evidence that suggests that the web-based interactive environment of the IVETs better engages students in the home setting, adding to the flexibility in how they are used as well as providing an important tool for providing more personalized instruction.
An important outcome, which was our second RQ, missing from this study is the impact of the IVET on students of different demographics, including subgoups of women and under-represented minorities. Although the data outcomes in Table III follow the same pattern for all groups, where the IVET treatment outperforms video-only, and both provide better learning outcomes than no treatment, the sample sizes of the sub-groups are too small to make any claims at this time about relative impact for each. Data will continue to be collected to increase the sample size of these sub-groups.
The Torque and Rotation IVET is just one example from our collection. We have now evaluated several IVETs for impact on students' development of problem-solving abilities, and the results are promising [16]. In addition, student surveys given after completion of multiple IVETs indicate that most students like the IVETs and feel that they helped them learn.
Our collection includes eight IVETs with plans to finish 22 more, providing an IVET for every chapter covered in the typical introductory physics course. Links to sample IVETs can be found at <https://ivet.rit.edu/IVET>. We expect the IVETs to be readily adopted by other instructors as we based their design on the work of Dancy and Henderson, who found that common reasons instructors cite for not adopting a research-based instructional strategy (RBIS) include the time required for instructors to implement the materials, the class time needed for their use, and the perception that RBISs are not closely enough aligned with the instructor's teaching methods and personal experiences [17]. Therefore, the IVETs have been developed to be freely available (unlike similar products provided by learning management systems or through publishers), they require no instructor training, and they take no class time because they can be assigned as homework simply by providing students with a link. Additionally, the problem-solving strategies promoted within the IVETs are those typically used by experts, and the problems the IVETs are based around are common examples used in introductory physics to illustrate specific concepts, making them more likely to be recognized as "good" or relevant problems by instructors.