Introductory physics students ’ retention of conceptual understanding of forces

A goal of introductory physics courses is to improve students’ conceptual understanding of forces and assist them to obtain a Newtonian framework on which they can build further understanding. In both algebraand calculus-based introductory physics sequences, students completed the Force Concept Inventory (FCI) at the beginning and end of the first course (mechanics) and then again at the beginning of the second course (electricity & magnetism, optics). Since some students delayed taking the second course, a wide range of time gaps exists between the two courses with no intervening instruction in physics. We use a database containing six years of assessments to investigate how retention of conceptual understanding depends on time. We also examine whether the completeness of the Newtonian framework affects retention of force conceptual understanding.


INTRODUCTION
The issue of students' retention of physics knowledge and conceptual understanding has been investigated by a number of studies including investigations in mechanics [1], electricity and magnetism [2], and quantum mechanics [3].For student learning to be effective as preparation for further learning or for any future application, it must persist beyond the end-of-semester exam or diagnostic.Therefore, assessment of the success of any particular teaching strategy using an end-of-semester diagnostic is valid if the end-of-semester diagnostic is correlated with long term conceptual understanding.
Francis et al. [1] investigated the retention of mechanics conceptual understanding for 127 algebra-based physics students using the Force Concept Inventory (FCI) [2] and found retention rates of 83% to 97% after one to three years.Pollock [3] studied a similar number of physics majors who progressed from a freshman physics course covering electricity and magnetism to a junior level course on the same subject.Using the Brief Electricity & Magnetism Assessment [4], he observed no statistical difference between students' scores at the end of the freshman class and the beginning of the junior-level class.Deslauriers and Wieman [5] used the Quantum Mechanics Conceptual Survey (QMCS) [6] to examine retention 6 to 18 months after completing a modern physics course for 73 engineering physics majors.They report only a few percent decrease in QMCS scores after the gap time, independent of whether instruction was traditional or used interactive engagement.
In this study we expand on the work of Francis et al. [1] and report on retention of mechanics conceptual understanding for more than 1000 algebra-based physics students and over 700 calculus-based physics students after intervals from 1 to 33 months.

II. SETTING AND METHODS
Georgia State University (GSU) is a large, diverse, urban research university with about 25,000 undergraduates.GSU has large science programs but no engineering and offers two-semester introductory physics sequences with laboratory at both the algebra-and calculuslevel.
Students in algebra-based physics are 60% biology majors and about 25% non-science majors.Algebra-based physics is taught in both lecture-style format and an integrated lecture and lab format (SCALE-UP [7]).FCI scores and student demographics are not significantly different between the two formats.In this study, data are reported for both formats together.Further work involving more students is needed to enable statistically valid analysis of these two groups separately.
Students in calculus-based physics are 45% computer science majors, 25% chemistry majors, and 15% physics majors.Calculus-based physics is taught only in lecturestyle format.
Students in the first course of each sequence complete the Force Concept Inventory [2] at the beginning and end of the course.Students again take the FCI at the beginning in the second course in each sequence.Since students do not always proceed directly from the first course to the second, students enter the second course with a range of gap times.In this work, scores are analyzed from over 1000 algebrabased physics students and over 700 calculus-based physics students with gap times from 1 to 33 months for students taking courses between 2011 and 2016.In this paper, these data are analyzed to answer two research questions: 1. How well is mechanics conceptual understanding retained by students in these two course sequences over time.
2. Does retention depend on the score at the end of the first course (presumably indicating the completeness of students' Newtonian framework).

III. RESULTS
In reporting the results, FCI scores from the beginning of the first course will be referred to as FCI1, scores from the end of the first course as FCI2, and scores from the beginning of the second course as FCI3.Data are only used for students having all three scores and having consented to allow the data to be used for research.FCI scores will be reported as a raw number out of a maximum score of 30 or as percentage.
As shown in Table I, algebra-based physics students on average had scores of 46.7% at the end of the first course and 40.3% at the beginning of the second course, a drop of approximately two correct answers out of 30 questions.Calculus-based physics students showed a similar decrease from 58.6% to 52.2%.To address research question 1 regarding retention over time, the interval between FCI2 and FCI3 was calculated for each student as well as the ratio of scores, FCI3/FCI2, which we will refer to as retention percentage in this paper.The average retention percentage was 91.3% for all algebra-based physics students and 90.6% for all calculusbased physics students Retention percentage as a function of time is displayed in Figs. 1 and 2 for algebra-based and calculus-based physics sequences, respectively.The error bars in each case show the standard error and as may be expected get larger at longer interval times since fewer students contribute to those data points.
Retention percentage for algebra-based physics remains over 90% for the first 6 months but then shows a decrease to between 70% and 80%.In contrast, retention percentage for calculus-based physics students shows no clear trend with interval time but remains generally high.To address the second research question regarding whether retention depends on the completeness of the student's Newtonian framework, we analyze retention percentage as a function of FCI2, that is, the FCI score at the end of the first course in the sequence.For this analysis, the distribution of scores in each test is important.Shown in Figs. 3 and 4 are the score distributions for FCI2 and FCI3, respectively, for algebra-based physics students.FCI2 shows a bell-shaped distribution with an average score of 14.0 out of 30 (46.7%) and a standard deviation of 5.6 (18.7%).FCI3 also shows a bell-shaped distribution with an average score of 12.1 out of 30 (40.3%) and a standard deviation of 5.2 (17.3%).The correlation coefficient between the FCI2 and FCI3 distributions is 0.68. Figure 5 shows the average retention percentage calculated for each value of FCI2.As is apparent from the graph, average retention percentages are over 100% for low FCI2 scores and decrease as FCI2 increases.This general trend is expected in any test-retest situation due to regression to the mean.Since the FCI2 and FCI3 distributions are not perfectly correlated, the randomness of the student performance will tend to result in low scorers showing more improvement and high scorers less.For example, a student who guesses poorly on answers for which they are unsure or is having a "bad day" for any other reason at the time of FCI2 is likely to be having a better day or be a better guesser on the day of FCI3.Likewise, high performers on FCI2 will on average score lower on FCI3 than they would if the distributions were perfectly correlated.Rather than correct for regression to the mean, a model was fit to the data accounting for regression to the mean proportional to the uncorrelated fraction of the distribution (that is, one minus the correlation coefficient) and hypothesizing a fixed drop in scores from FCI2 to FCI3.The fitting is shown as a solid line in Fig. 5 and the best fit parameters for this model correspond to a decrease of 1.9 out of 30 (6.3%).The model fits the observed data well except at high values of FCI2 for which the number of students is small and the maximum score of 30 may decrease measured retention.Data were analyzed for calculus-based physics in a similar way.Figures 6 and 7 show the FCI2 and FCI3 distributions of scores, respectively.The distributions of FCI2 and FCI3 are considerably less bell-shaped for calculus-based physics than for algebra-based physics.The FCI2 distribution has an average score of 17.6 out of 30 (58.6%) and a standard deviation of 6.8 (22.7%).The FCI3 distribution has an average score of 15.7 out of 30 (52.2%) and a standard deviation of 6.9 (23.0%).The correlation coefficient between the two distributions is 0.81, significantly higher than for algebra-based physics.Figure 8 shows the average retention percentage calculated for each value of FCI2 for calculus-based physics.Figure 8 also shows the fit line for a model that includes a fixed drop in scores from FCI2 to FCI3 along with regression to the mean as previously described.Even though the FCI2 and FCI3 scores are not normally distributed for the calculus-based course, regression to the mean of the uncorrelated fraction would be expected.As Fig. 8 shows, these model assumptions appear to explain the retention fairly well across the entire range of values of FCI2.The best fit parameters for this model correspond to a decrease of 1.9 out of 30 (6.3%), exactly the same as was determined for the algebra-based course.

IV. DISCUSSION AND CONCLUSIONS
This investigation generally confirms the conclusion of Francis et al. [1] that student retention of mechanics conceptual understanding remains high over at least several years after completing the first physics course.This study also reveals more subtle effects, such as the apparent decrease in retention after about six months for algebrabased physics students while no such decrease is observed for calculus-based physics students.These results are similar to the high retention observed in other content areas of physics by Pollock [3] for electricity and magnetism conceptual understanding in calculus-based physics and for quantum physics concepts as observed by Deslauriers and Wieman [5].
This study also investigates the question of whether the completeness of the knowledge framework affects retention in the case of forces and mechanics.Once regression to the mean is factored in, the data presented here are consistent with a 6% drop in scores independent of the score at the end of the first course for both algebra-and calculus-based introductory physics sequences.This indicates that a more complete Newtonian framework does not result in higher retention of conceptual understanding.

FIG 1 .
FIG 1. Retention percentage versus interval between FCI2 and FCI3 for students in algebra-based physics sequence.

FIG 2 .
FIG 2. Retention percentage versus interval between FCI2 and FCI3 for students in calculus-based physics sequence.

FIG 3 .
FIG 3. Distribution of scores for FCI2 taken at the end of first course in algebra-based physics sequence.

FIG 4 .FIG 5 .
FIG 4. Distribution of scores for FCI3 taken at the beginning of second course in algebra-based physics sequence.

FIG 6 .
FIG 6. Distribution of scores for FCI2 taken at the end of first course in calculus-based physics sequence.

FIG 8 .
FIG 8. Diamonds represent the average retention percentage as a function of FCI2 score for the calculusbased physics sequence.The solid line shows the fit using a model that includes regression to the mean and a fixed decrease in score.Error bars show standard error.

TABLE I .
Numbers of students and average scores for FCI given at three points in introductory physics sequence.