Exploring the relationship between exam performance and student participation/engagement in introductory mechanics

We have examined the relationship between exam performance and participation in the lecture, recitation, laboratory and online homework components of large enrollment introductory mechanics courses populated mainly by first-year engineering students, one very large (1600 students) on a residential campus and one smaller (150 students) on a commuter campus. In both, we find a strong correlation between exam performance and recitation attendance tracked by clicker questions that begin and end each recitation session. This is encouraging since the recitations are designed to engage students in active learning by solving multipart problems that highlight essential concepts. We also find correlations between exam performance and student engagement in the lecture, laboratory and online homework components of these courses. We argue that the nature of the latter correlation offers an opportunity for action to improve course DFW rates since online homework data flows in steadily from the very beginning of each semester.


I. INTRODUCTION
Many post-secondary instructors associate higher student attendance with better exam performance and thus a higher grade in the course.To examine if this is truly the case and to determine the extent to which attendance predicts course grade, Credé, Roch and Kieszczynka (CRK) conducted a meta-analysis of 68 different studies (N= 21,164) correlating course attendance and course grades.CRK showed that course attendance is the strongest of all possible predictors of course grade.
While this may seem like a definitive answer, CRK only included 11 studies (N=8524) examining this correlation between course attendance and course grade in science courses.Due to the sampling procedures utilized in this study the majority of these science courses are in life and medical science.[1] In this study we extended this existing research to introductory calculus-based physics courses and attempt to answer the following research questions.
Research Question 1: Does class attendance in the different aspects of the calculus-based introductory physics course (Lecture and Recitation) correlate with course exam scores?
Research Question 2: Does this relationship exist in both large enrollment courses at a major Research level 1 university and in a smaller urban non-residency university?
Research Question 3: Can the results of these correlations be utilized to identify poorly performing students early in the semester?
The answers to these questions may provide insight into how to improve performance in the introductory physics course.Such courses are typically a requirement for first-year engineering students in order to gain entry to the engineering program.Consequently, failure to pass this course causes a significant increase in time-to-degree or a change of degree objective, as well as an additional financial burden to the student.It is therefore no surprise that deans, department heads and instructors tend to pay close attention to the DFW rate (letter grades D or F, and withdrawals) in these courses.In this paper we discuss insights gained from our investigations of the above research questions.Specific interest is paid to the characteristics of students in two categories -those who succeed and those who do not succeed in achieving at least a C-grade.

II. METHODOLOGY
To inform our continuing efforts to improve instruction and reduce DFW rates, we examined correlations between exam performance and participation in various components of two introductory mechanics courses that were informed by physics education research.For each student and each course component, we examined the correlation between the number of absences and the exam performance.We use the participation points as measured by the use of clickers as a surrogate for attendance.

A. Introductory Calculus-Based Physics at Purdue
One course chosen for this study was PHYS 17200 at Purdue University during the 2016 spring semester.The vast majority of the 1614 enrolled students were in the first-year engineering program.Approximately fifteen years ago, we adopted Matter & Interactions [2] as our introductory mechanics text.The weekly PHYS 17200 course structure consists of two 50-minute lectures during which 3-5 clicker questions are posed (worth 5% of the total grade), one 50-minute recitation during which students work problems in groups of four on whiteboards (15%), a two-hour laboratory (15%), and online homework following each lecture (15%).Recitation sessions at Purdue begin and end with clicker questions to engage groups of four students in producing white-board solutions to challenging problems aligned with material covered in recent lecture and laboratory sessions.Dedicated online homework problems exploring laboratory results reward students for engaging with computational and hands-on experimental work during laboratory sessions.Attendance is taken via clickers in lecture and recitation and by hand in lab.The remaining 50% of a student's grade is determined by the performance on 3 one-hour exams (10% each) and a two-hour final exam (20%).The course syllabus sets the grade cutoffs as follows: A: > 89%, with the remaining grades in 10 percentage point intervals, e.g., a B is between 79% and 89%, and an F is less than 59%.We purposely set a high bar for grades in the syllabus, but at the end of the course, grade cutoffs are lowered on a sliding scale with the greatest lowering occurring for an A (by about 6%) and the least lowering for a D (by less than 1%).We know from past experience that adherence to the original grade cutoffs would result in an unacceptable pass rate in the course.

B. Introductory Calculus-Based Physics at IUPUI
The second course we studied was a similar course at another campus in the Purdue system, PHYS 15200 at Indiana University-Purdue University Indianapolis (IUPUI).In contrast to the West Lafayette campus, IUPUI is primarily a commuter institution.Approximately five percent of IUPUI students live on campus.IUPUI students are on average older and more likely to be first-generation students.PHYS 15200 also a calculus-based introductory mechanics course with an enrollment of 150 students, using Tipler and Mosca Physics for Scientists and Engineers [3].The course structure is similar in structure to the Purdue course except that there are two recitations each week, and the weighting of non-exam assignments is slightly different: homework 15%, JiTT (Just In Time Teaching) [4] assignments 10%, lecture clicker questions 5%, lab 15%, midterms 30%, final 20%.In addition, PHYS 15200 had five mid-term exams while PHYS 17200 had three.

C. Course Commonalities
Lectures in both courses include clicker questions in a peer instruction mode [5] in order to engage students.Students are encouraged to discuss their answers before voting on an answer.In both courses, exams are a mixture of multiple choice and free response problems.In addition, the Purdue course also features challenging problems worked through in real time with input from students.Online solutions of the problems are posted afterward.Both courses use WebAssign for online homework [6] and both had a comprehensive final exam.

A. Research Questions 1 and 2
The answers to research questions 1 and 2 are evident in our primary result -a dramatic correlation between exam score and all measures of attendance, both lectures and recitation.See figure 1, where the error bars represent the error in the mean.The parameters of the fitted linear equation are presented in the inset, and the chi-squared values are per degree of freedom.These correlations are observed in both courses studied in spite of the significant differences in student population.The quality of the data did not warrant a higher order fit.
There are notable differences between PHYS 17200 and PHYS 15200 in the dependence of exam score on attendance.This can be characterized on the basis of the slope of each fit in figure 1.The fitted slope, "M2", is a measure of exam points lost for each class skipped.At Purdue, M2 is larger for a recitation than for lecture, indicating that recitations had a greater impact on preparation for exams.In contrast, the opposite is observed for IUPUI.We speculate that this may be due to the greater number of recitations per week at IUPUI than at Purdue.It may also be due to the nature of the exam questions in PHYS 17200: we deliberately construct problems asking for different predictions about physical systems that students have analyzed in recitation or asking for predictions about different physical situations that require reasoning and analysis with parallels to solutions of recitation problems.

B. Research Question 3
Because of the much larger enrollment in the Purdue class, we can do some additional analysis in an effort to understand the characteristics of students who earn a D. Given our grade cutoffs, a student with a 90% non-exam score, which happens to be the class average, needs only about a 50% exam score to earn a C-grade.We wish to understand why so many students earn a D under these circumstance.In order to do this we have examined the differences between the performance of students who received D grades and and A, B, or C grades and between those with D grades and C grades.
As Table I shows, D students consistently underperform the students earning A, B or C grades in all course components but especially in exam scores which is indicative of the role that lectures, recitations, homework and labs play in preparing students for exams.Note, this homework score includes contributions from online post-laboratory problem sets.Figure 2 illustrates the difference in performance on the non-exam components for each grade group and shows a significant fall off between the D students and the others.We note specifically that their average homework score of 83.6% is significantly lower than the 94.2% of the A, B or C recipients.In Table II, we display the D student and C student homework performance over time.Homeworks 3-7 and 8-16 precede exams 1 and 2, respectively.We have omitted the few post-lab homework scores in this tabulation.The D student deficit is evident from the very beginning of the course.This observation provides an answer to research question 3 and suggests that there is an opportunity for an early intervention to promote engagement of students who are at risk.

V. CONCLUSIONS
In spite of the fact that the two introductory courses analyzed here use different texts, slightly different formats, and different instructors, we find a very similar relationship between missed recitations and lectures and exam performance.Our study therefore confirms and extends the results found by CRK.[1] Our examination of the students receiving a grade of D compared to those receiving a C or better indicates that the former lose points on each course component.It is partic- ularly troubling to us as instructors that the D students are underperforming on the homework, in spite of considerable resources available to assist them, such as access to a Help Center, faculty office hours, as well as availability via email in the evening.
We acknowledge that our study has its limitations.We do not know why students are absent from class, nor do we know why students who receive a D underperform on the homework.Both of these issues are left to future investigations.There is some evidence [7] that participation by students in a course-focused social media platform such as CourseNetworking are more engaged in the class.Thus, the underperformance of the D students may be dominated by insufficient engagement rather than an inherent lack of ability.We have examined the participation level of predominantly firstyear students in multiple components of the introductory mechanics course at two different institutions.We find a common trend: the more recitations, lectures, homework or labs skipped by a student, the greater the likelihood of an underperformance on exams.While this is not totally unexpected, it does suggest that by monitoring this non-performance early in the semester, a course instructor may, by initiating an early intervention, reduce the number of students who would otherwise receive a D.

FIG. 1 :
FIG. 1: Mean exam percentage vs number of recitations and lectures skipped at Purdue (a,b) and IUPUI (c,d).

TABLE I :
Comparison of D students to C or better in PHYS 17200 at Purdue University.

TABLE II :
Comparison of homework performance of D students to C students in PHYS 17200 at Purdue University.HWK 3-7 HWK 8-16 HWK 17-24 Overall percentage