Equity of success in CLASP courses at UC Davis

We have recently described the reformed introductory physics course, Collaborative Learning through Active Sense-Making in Physics (CLASP), for bioscience students at UC Davis and argued that the course was more successful than its predecessor (Physics 5) by several measures. Now we examine the effects of these courses for different student ethnic groups. We find that, compared to Physics 5, students of most ethnic backgrounds were more successful in CLASP. We also find that students from ethnic groups underrepresented in STEM who took the CLASP course were more likely to graduate as STEM majors. We discuss possible features of CLASP that might explain these results.


I. INTRODUCTION
Over the past 30 years there have been many physics course reform efforts [1][2][3][4][5] and they have nearly all featured active learning via small-group peer-peer discussions as a major course component.In practice these peer-peer discussions often take place in a studiotype setting where students work together in small groups on activities facilitated by the instructors.These types of courses are interchangeably referred to as interactive engagement and active learning courses.
The Physics Education Research (PER) community and the greater Discipline-Based Education Research (DBER) field have provided overwhelming evidence that active learning courses on average improve learning outcomes for students [6][7][8].However, research has shown that there is a huge variety in student performance on various metrics between different demographic populations [2,9].For example, in their 2014 paper, Eddy and Hogan discuss how different populations behave and perform differently in active learning classrooms suggesting that there is a more nuanced answer to the question, "Is active learning better?" [10].They find that while all students performed better in an active learning environment, the gain in exam performance almost doubled for African American students and improved similarly for first generation college students.The African American students had different perceptions of the class than other groups, suggesting that the mechanism for increased performance may not be the same for all groups of students [10].Therefore, they suggest that it's important to consider for whom active learning is better, and by which metric.
Studies have shown that grades are not necessarily good indicators of learning because students have an ability to solve physics problems without a good understanding of the physics content [11][12], therefore student pass-rates may not be a good indicator of learning goals.However, grades still play an important role in the academic process.For example, because † Deceased 8 January 2017.
grades represent an official form of evaluation, good grades can encourage students, and poor grades can deter them [13].Furthermore, pass-rates are a very real indicator of college success, as course failures can mean students need to repeat coursework, drop out of their major, and/or leave college entirely.

II. RESEARCH QUESTIONS
In this paper we examine several years of transcript data for students across different ethnic groups in order to better understand how the switch from a traditional course to the active-learning based Collaborative Learning through active Sense-making in Physics (CLASP) curriculum has impacted different groups of students.We have previously shown that CLASP produces better learning gains on average than the traditional course it replaced (Physics 5) [1].As a part of a larger project, in this study we consider equity across student ethnicity by examining course grades and pathways towards graduation.We examine the success of students grouped by self-identified ethnicity (as defined and collected by UC Davis), comparing success in CLASP with that in Physics 5.The practice of examining improvements within specific student populations, without comparing their success to that of some normative group, is referred to as applying a model of Equity of individuality [14].We say more about how this model is used in our discussion.
We investigate the following research questions: 1.How do students of different ethnic backgrounds perform in the CLASP series as compared to the traditional Physics 5 course?2. How does taking CLASP affect STEM graduation rates for students of different ethnic backgrounds?

III. COURSE SETTING
As described in previous work [1], the CLASP curriculum is an introductory physics course required for many STEM degrees, including the biosciences.2017 PERC Proceedings, models (including words, graphs, equations, etc.) and using these models in qualitative and quantitative analysis of real-world physical, chemical, and biological situations.The active learning elements of the course are carried out in studio-type discussion/laboratory (DL) sections of 25-35 students.These DL sections include small group activities where students work together to understand a model and practice applying it before engaging in whole-class discussions.The Physics 5 curriculum is a typical traditional intro physics course with standard lecture and lab components.Both Physics 5 and CLASP are three-course series, meaning there are Physics 5A, 5B and 5C courses as well as CLASP A, CLASP B and CLASP C. Each series is meant to be taken in sequence, one course per quarter.

IV. METHODS
The UC Davis Physics Department began offering CLASP in 1995, and completely replaced the traditional lecture/lab course by academic year 96-97.This switchover creates the conditions of a natural experiment that allows us to compare student successes in lecture/lab courses (Physics 5) with successes in an active-learning studio-type courses (CLASP).
For both research questions we consider the same population of students, comparing those who took Physics 5 to those who took CLASP.There was a small overlap in time between two trial runs of CLASP and the last of the regular Physics 5 courses so there might be slight selection effects.Selection effects must be limited since only about 5% of all the students in our data had a choice to enter a CLASP trial.Average UC Davis GPAs of students who chose to enter CLASP trials was 2.90 compared to 2.96 for the contemporaneous Physics 5 students (effect size of 0.1).Because the data from each course were collected in different years, we need to consider larger institutional changes that might impact our results.In Fall of 1993, UC Davis changed the procedure for registration and subsequently the rate of students dropping introductory STEM courses decreased by a factor of 3.8 (from 18.2% to 4.8%).Therefore, we limited our sample to students who took physics after the 92-93 academic year.We limited the number of years included after the switch to CLASP in order to compare similar student groups.We selected 99-00 as the last academic year in our study giving us 3 full years of Physics 5 and 4 full years of CLASP.
The ethnicity data are self-reported by all students from pre-determined ethnicity categories created and named by UC Davis.Whether these categories truly represent the students' ethnicities warrants further investigation beyond the scope of this study.Students were only allowed to select one option but they could choose not to select any.The student population represented in the data is 84% US citizens and 16% permanent residents.Ethnicities appearing in the data with fewer than 80 total students enrolled in CLASP over 4 years were excluded from this study due to the uncertain meaning of statistical analysis of small groups.However, if a group that would have been excluded by this cutoff is considered underrepresented in physics [15], then they were included.Groups considered underrepresented minorities (URM) in physics in this study include African American, Latina(o), Chicana(o) or Mexican American, Hispanic-other, Native American, and Pacific Islander students.The population fitting the URM classification is about 89% US citizens and 11% permanent residents.Table I names the student ethnic groups and gives the populations of each group in each of the course series data sets.

V. R1: COURSE SUCCESS
We use logistic regression to predict the odds ratio for passing (odds of passing CLASP / odds of passing Physics 5) the first two parts of the course (ex: CLASP A & B) without failing either.We leave out part C of both courses because many majors require only A and B. The odds of passing are equal to the probability of passing divided by the probability of failing.The model we use controls for the GPA each student had upon entering their first physics course because GPA is a significant covariate.The resulting odds ratios are given in Table I.In Fig. 1 we compare the fraction of students in each ethnic group who aren't successful in Physics 5 with that of CLASP.Students are unsuccessful if they receive a grade less than C-or withdraw from the course.In all cases where there is a significant difference between the two courses, students are more likely to be unsuccessful in Physics 5 than in CLASP.

VI. R2: STEM DEGREE SUCCESS
Because there is a significant observable improvement in pass-rates for all students (and for students from URM ethnic groups in particular) for those who take the CLASP series of courses instead of Physics 5, we explored if there were any long-term effects of this improved success rate.Again we use logistic regression to predict the odds ratio of success, this time limiting our sample to students who started either course as a STEM major and defining success as graduating with a STEM degree.Table II shows that students who take CLASP have 1.31 times greater odds of graduating with a STEM degree than those who take Physics 5, and students from URM ethnic groups have odds 1.46 times larger if they take CLASP.Again GPA is a significant covariate and is used in the model.Figure 2(a) shows how the fraction of students who leave STEM (start in STEM but change major or don't graduate) differs by ethnic group.For every ethnicity where the probability of leaving is noticeably different between CLASP and Physics 5 (FP, LA, MX, WH and URM) the impact of CLASP on graduation rate is positive.In every other measured case (with the exception of students of East Indian ethnicity) the positive trend exists but is not significant.
For our final analysis, we considered only those students who were successful in Physics 5 or CLASP (success defined as earning >D+ in the first course in each series), and examined the fraction of those students who left STEM (see Fig. 2(b).).Because CLASP has an overall higher pass-rates than Physics 5, one might expect that on average, successful students in Physics 5 were 'better' students in some way than successful students in CLASP.If this were true, one might expect successful Physics 5 students to be more likely to remain in STEM than successful CLASP students.However, when we directly compare these groups, we find that for students of most ethnicities the opposite trend exists, suggesting that the higher pass rate in CLASP does not merely reflect giving out higher grades to unprepared students.Because we are only considering successful students, our sample size and thus our statistical power has decreased, but we still notice a difference for students from ethnic groups underrepresented in physics.Students from URM ethnic groups who are successful in CLASP are more likely to remain in STEM and graduate with a STEM degree than those who are successful in Physics 5.

VII. DISCUSSION AND CONCLUSIONS
Our results indicate that students are more likely to pass CLASP than the traditional Physics 5 series.By using a model of Equity of Individuality to frame our analysis, we claim that implementing CLASP is an equitable practice as it resulted in increased success for most student ethnic groups and no significant decreases for any groups.Nevertheless, it's clear that neither gains nor the overall success measures are equivalent for all groups.These comparisons inform other models of equity which we are not applying, but could be the subject of future analyses.
We also show that taking CLASP improves the chances of students from URM groups in physics of graduating with a STEM degree.Finally, our analysis also indicates that something about the CLASP series, beyond just passing the course, impacts students from URM groups in a way that retains them in STEM more than the original physics series did.
However, there are also outside influences that may differentiate the two groups that have nothing to do with Physics 5 or CLASP.For example, UC Davis registration changed from in-person to phone in 1993, and internet registration was added in 2000.Initial analysis indicates that the 1993 change had a significant effect on the drop rates for these courses.It is possible This paper tells only a partial story regarding equity and success in CLASP.Pass rates for students in CLASP have changed in recent years, indicating that some important elements of CLASP are tentative, and caution must be used before making claims that CLASP curriculum alone is responsible for improved success in physics and retention in STEM.
Even if we operate under the assumption that CLASP does improve student retention in STEM, we can currently make few claims about what aspects of the CLASP curriculum contribute to these outcomes.However, there are several potentially fruitful avenues to pursue to explain our results.For example, since CLASP students are actively engaged in model construction and application instead of passively listening in lecture, they have more opportunities to successfully talk about science and use scientific tools, moreover, these opportunities happen in a social environment where students are communicating with instructors and peers.This CLASP feature aligns with what research says is important for self-efficacy [16], an important trait for persistence in STEM.
Future work by our group will examine how these potential factors that contribute to CLASP success.We will also examine how our results change with time, paying particular attention to grading practices in the CLASP and Physics 5 series and situate our findings in relevant race and identity theories edited by Ding, Traxler, and Cao; Peer-reviewed, doi:10.1119/perc.2017.pr.068Published by the American Association of Physics Teachers under a Creative Commons Attribution 3.0 license.Further distribution must maintain attribution to the article's authors, title, proceedings citation, and DOI.

TABLE I
Fraction of students in the most populated ethnic groups who did not drop and received a grade less than C-.Error bars represent standard error.

TABLE II
that UC Davis STEM students in the late 90's were statistically more successful than those in the early 90's.