Relevance and responsibility : preliminary results from implementation of a cooperative problem solving model in a large introductory physics course

Interactive instruction make measurable gains in concept learning, but problem-solving skills and develop-ment of a scientific mindset are often missed in the large lecture environment of introductory physics courses. We describe an implementation of the Cooperative Problem Solving model (Heller & Heller 2010) for over 400 students in an Introductory Mechanics course for Life Science Majors at UC San Diego, aimed at improving problem-solving, relevance and student collaboration in learning. We describe our flipped-model approach with 25 hours of video lectures, problem-solving skill development through training worksheets and strategy scaffolds, and bi-weekly team projects combining physical and life science topics. Nine sections of this course were conducted alongside a large lecture control, all taught by the same instructor. We discuss comparisons in student performance on exams and the Force Concepts Inventory, and demonstrate clear gains in persistence in this and subsequent Physics courses. We also examine students’ use of the video lectures.


I. INTRODUCTION
Interactive instruction in large survey physics courses often focuses on students' conceptual learning and addressing misconceptions [1].Problem solving skill development may be relegated to separate discussion sections or not addressed at all.Yet the ability to dissect complex problems, identify resources, apply logical reasoning and mathematical analyses, and check that an answer makes sense are skills students often struggle with but can benefit non-physics majors in their other studies and future careers [2].
Several studies have investigated approaches to problem solving skill development through cooperative learning models, in which students work together in small groups as a means of "maximiz[ing] their own and each other's learning" [3].Implementations of cooperative learning in engineering programs have demonstrated improved attitudes toward learning, increased persistence in STEM programs, and greater understanding of the collaborative nature of scientific work [4].In physics, gains in the Force Concepts Inventory (FCI; [5]) are shown to be as good or better than interactive lecture models and far superior to traditional lecture [6].In addition, there is evidence that female students find traditional large lectures alienating, overly competitive and prohibitive of inquiry, and have a greater preference for group learning than their male peers [7][8][9] Here we describe an adaptation of the Cooperative Problem Solving (CPS) model of [6] for a large introductory mechanics course for life-science majors at UC San Diego using a "flipped" format; i.e., with lectures provided online, and problem solving the primary focus of class time.The model was explored in order to address what the lead author has found to be shortcomings in students' development of both problem solving skills and a physics/scientist mindset in large lecture environments.The goal of the experiment was to evaluate whether a CPS approach could better develop quantitative, scientific and growth mindsets; help students learn to de-velop and follow procedures for solving complex problems; improve their skills in working on and among collaborative teams; find contexts for physics in both their majors and the "real world"; and communicate process and solutions for scientific and quantitative problems to peers and experts.

II. IMPLEMENTATION
Physics 1A is the first-quarter, lower-division course on Newtonian mechanics designed for life-science majors, and is part of a three-quarter series (Physics 1B = Electricity & Magnetism, Physics 1C = Optics & Modern Physics).It is a mixed algebra-calculus course traditionally taught in large lectures (300-400 students).There is a parallel laboratory course taught by separate instructors, so the lecture course is traditionally concept-driven and theory-based.A single graduate teaching assistant (TA) is generally assigned to lead discussion sections in which most problem-solving training takes place.However, the resulting student:instructor ratio of 150-200:1 means little one-on-one interaction occurs outside of office hours.Even when interactive learning techniques are deployed to increase student engagement and performance (e.g., peer learning, lecture tutorials, in-class writing, metacognition [10][11][12]), the scale of the course inhibits meaningful attention to individual students' needs and limits the development of quantitative analysis and problem-solving skills.The associated laboratory course, while having a lower student:instructor ratio (20-30:1), focuses on laboratory skills over quantitative problem solving.
In Fall 2015, the lead author was assigned three lecture sections of Physics 1A.Two of the lecture sections (≈450 students) were reorganized into nine CPS-based courses of ∼50 students each, meeting three hours per week ("Workshop").The largest lecture section (≈360 students) was retained as a control case ("Lecture") and taught as an interactive lecture, also meeting three hours per week.The number of gradu-ate TAs was doubled from three to six.They led multiple discussion sessions (open to all students), and at least two were present at every class meeting to facilitate peer learning (Lecture) or problem solving (Workshop).In the Workshop sections, the faculty instructor and TAs had comparable roles during in-class instruction, including leading class discussions and evaluating student work.All students took the same bi-weekly and final exams, which contributed the same fraction to their final grade; and were assigned online homework, although Workshop students had assignments that were reduced in length by 30-60%.Finally, all students completed weekly metacognition writing prompts, aimed at drawing out self-reflections on their learning [10,13], and earned participation credit through peer instruction (Lecture "clicker" questions) or straight attendance (Workshop).The ten-week quarter was divided into five two-week cycles covering Kinematics, Forces, Energy, Momentum, and Rotation.

A. Online Video Lectures
Workshop video lectures were delivered through the online course management system [14].70 videos totalling 25 hours were produced during the summer preceding the course with UCSD's Educational Technology Services (ETS).The videos were organized into "concept" and "problem solving" formats, the latter using Learning Glass technology [15].The videos could be watched in any order or pace, but had to be completed in advance of semi-weekly online quizzes to assure class preparation (Lecture students took equivalent prelecture online quizzes).Video viewing statistics, including individual student access times and dates, were recorded.

B. Student Teams
Students in the Workshop sections were organized into teams of up to five students.Team assignments were initially randomized; in subsequent cycles students were grouped following a "cohort" model, with even distribution of high-, middle-and low-performing students based on the previous cycle's exam.The purpose of the student teams was to (1) encourage students to work collaboratively in class on training sets and establish group study habits; and (2) complete teambased projects focused on complex problems.Team work was facilitated by the structure of the classroom, a flexiblefurniture room with moveable tables and chairs, and video screens and chalk/white boards on all walls.Teams were also assigned "particle-zoo" mascots to provide some early exposure to modern physics concepts (covered in Physics 1C).

C. Training Sets
Class time was primarily used to work through "training sets", worksheet tutorials that guided problem solving skill development.Initial training sets were designed following the lecture tutorials of [12], but subsequent training sets led students through associated Problem Solving Process Guides: learning scaffolds with step-by-step instructions on how to solve types of physics problems (e.g., "Solving Force Problems", "Solving Collision Problems").For example, a training set sequence for a work-energy problem, such as determining the height of a swinging pendulum, would guide students to (1) write down known values, identify the target quantity and make a prediction for the solution; (2) identify forces acting on moving objects and their nature (conservative, dissipative, constraint); (3) create an initial/final energy table, writing down expressions/values for kinetic and potential energies, and sum to total energy; (4) if dissipative forces are present, draw a force diagram and compute (or write an expression for) work done by the components of these forces parallel to the direction of motion; (5) set dissipative work equal to change in total energy and solve for unknowns; and (6) check solution units and compare to initial prediction.
Examples illustrating these steps were presented in both the problem solving video lectures and in class.Students then worked with their teams to complete one or more follow-on examples.Instructors circulated around the room to help individual students and student teams, and solutions were subsequently reviewed either by the instructor or (more commonly) through student "share-outs", depending on the difficulty and degree of completion.Some portions of the worksheets were left to be completed by students on their own.Completed worksheets were neither collected nor graded, and solutions were distributed at the end of each cycle as a study resource.

D. Team Cycle Projects
To align physics content learning with students' life science academic and career interests, we developed a series of bi-weekly projects, in consultation with Physics and Biochemistry faculty, based on the concept focus of a given cycle but with specific biomedical or life science applications.These projects were designed to be more complex and openended than either the training set or homework problems, and required both background research and analysis of real or simulated data which differed between the teams.Cycle 1: Tracking Fruit Flies (biomechanics): Using individual 2D time-sequence velocity measurements, teams reconstructed the path of a confined fruit fly, and determined if it was attracted to or repelled by a chemical in the confined area.Cycle 2: Physics and Lawsuits (physical science in law): Teams assessed the circumstances of a collision between two cars in the vicinity of a blind curve (differing curvature, road conditions and posted speed limit), and assessed whether the driver lied and if they or the city (or both) were at fault.Cycle 3: Powering a Rural Hospital (physics & medical policymaking): Teams determined if energy needs of a rural hospital (varied sizes) could be provided by an on-site hydroelectric plant at the base of a waterfall (varied flow rate, height).
Cycle 4: JFK's Assassination (forensic science): Teams reviewed the 1963 Zapruder film of JFK's assassination and analyzed two models of head reaction [16] using 1D momentum conservation to determine if a second shooter was necessary.
Within the teams, students rotated through five project roles-Manager, Researcher, Solver, Communicator and Skeptic-expanded from [6].Grades were based on team reports and in-class presentations (cycles 1 & 2) evaluated by graduate student and faculty instructors; and peer evaluation.Students were provided grading rubrics in advance, and starting in cycle 2 were shown examples of projects that met rubric guidelines.Presentations were eliminated in cycles 3 and 4 to reduce student workload (see below).

E. Additional Assessments and Surveys
Students were assigned the FCI and Colorado Learning Attitudes about Science Survey (CLASS; [17]), both preand post-course, to assess changes in basic Physics concept knowledge and perceptions of physics.These were conducted online, and a small amount of extra credit was given for completing each on time.In addition, optional (no credit) online surveys aimed at gauging students' perceptions of the course and course elements were sent out on a bi-weekly basis for the Workshop and midway through the quarter for the Lecture.These and the metacognition prompts were intended to provide insight into the students' attitudes about the course.

III. PRELIMINARY RESULTS
What Impact did the Cooperative Problem Solving Model have on Student Learning?We examined differences in student exam scores and FCI gains (g; Figure 1) between the Lecture and all Workshop students, and among the individual Workshop sections.Only the first exam showed a statistically significant difference, with Workshop students performing significantly worse (averages of 65% versus 72%, p < 0.01).We attribute this to an excessive workload at the start of the term, which was made clear to us in student surveys.Once corrected, subsequent exam performance, including the final, was statistically equivalent.We also examined students' performance on types of exam questions, and found that the Workshop students performed marginally better on complex problems (p = 0.08, cumulative over all exams) but significantly worse on pure concept questions (p < 0.01).
The average FCI gains for students who completed both pre-and post-FCI exams were indistinguishable between the Lecture (0.377, N = 208) and Workshop (0.387, N = 182; p = 0.88).Both gain measures were on par with published measures for interactive lectures and partial CPS implementation, and well above average gains for traditional lecture courses.What Impact did the Cooperative Problem Solving Model have on Student Persistence?After early drops by students from all sections at the second week Course Add deadline (see video viewing statistics in Figure 2), Workshop students dropped the 1A course at about half the rate (468 to 453, −3.2%) as Lecture students (361 to 340, −5.8%).This is a statistically significant difference based on a binomial test (p = 0.013).We also tracked the enrollment and course grades for students in the subsequent Physics 1B and 1C courses in Winter and Spring 2016.Among the Workshop students, 32.5% (N = 259) completed 1B and 26.8% (N = 213) completed both 1B and 1C.Both of these fractions are significantly higher (p < 0.01) than those for the Lecture students, 23.1% (N = 184) and 19.0% (N = 151), respectively.However, subsequent course grades were statistically equivalent.How Did Students Use the Video Lectures?Viewing statistics (Figure 2) peak at the start of each cycle and just before an exam; and between 3pm-8pm Sunday through Friday, cutting off at the start of exams.Overall, fewer than half of the students watched more than half of the videos, with an average of 28.6 unique videos viewed per student.Students watched on average 40.6 videos, implying an average video repeat rate of 1.42.Concept videos were viewed more often than problem solving videos.These patterns suggest that students used the videos as study materials in much the same way as more traditional resources (notes, practice quizzes, etc.).We have not yet correlated viewing statistics with exam performance.

IV. SUMMARY AND CONCLUSIONS
Our preliminary analysis shows evidence of modest gains in complex problem solving skills but declines in conceptual learning in our implementation of the CPS model.However, overall performance in traditional assessments (multiplechoice exams) is equivalent between CPS and Interactive Lecture models, and we found no significant difference in FCI gains (which are higher than traditional lecture) or subsequent Physics 1 course grades.We do find a significant difference in students' persistence both in the course itself and throughout the Physics 1 series.While this result suggests that persistence may be tied to factors other than assessed knowledge, possibly relevance or agency, it is necessary to analyze students' attitudes as gauged by the CLASS and inclass surveys.While we observed several other learning gains for the Workshop students, such as better adherence to evaluation metrics (i.e., steady improvement in team write-up and presentation scores) and the application of higher-level skills (e.g., analysis of quantitative data, physical reasoning, etc.) that we unfortunately had no assessments to adequately measure or compare gains.Quantitative evaluation of these skill developments must be examined in future studies.