Characteristics of institutions with Learning Assistant programs: An equity investigation

Learning Assistant (LA) programs support instructors in transforming their courses to use evidence-based instructional strategies. We investigated the types of institutions that have LA programs to better understand how the distribution of those programs supports education across institutions. Using public websites and databases, we compared schools with and without LA programs, looking at a variety of institutional characteristics to determine whether the distribution of LA programs is equitable across different types of institutions. The results indicated that the majority of LA programs were at large-enrollment, public, research-intensive, four-year institutions. Minority serving institutions (MSIs) were equally well represented in the institutions with LA programs. MSIs with LA programs were, however, more likely than non-MSIs with LA programs to be teaching-intensive or two-year colleges. Further expanding the LA model into teaching-intensive and two-year colleges will more equitably share the benefits of the LA model with students from minoritized groups.


I. INTRODUCTION
Evidence-based instructional strategies often have a goal of improving equity in student outcomes. For example, several studies have investigated equity across a range of student outcomes for active-engagement strategies [1][2][3][4][5][6][7]. These studies have largely focused on implementations at a single institution. This local focus leaves open the possibility of a strategy that supports equity locally but increases inequity nationally through unequal access to that strategy [8].
In physics education research, researchers seldom define equity [1,[9][10][11] and they often treat equality of learning as an equitable outcome, see for example [1,[9][10][11]. Under equality of learning, an evidence-based instructional strategy is considered equitable if it improves outcomes for students from dominant and minoritized groups equal amounts, which maintains inequities. If such a strategy is disproportionately used at institutions that overrepresent students from dominant groups, then it will increase inequities at a national level. For example, two-year colleges disproportionately serve firstgeneration and students of color [12] but are nearly excluded from discipline-based education research literature [13,14].
In the present research, we focus on the learning assistant (LA) model. The LA model provides structure to support instructors in using evidence-based instructional strategies that fit their curricular needs [15,16]. An LA program is a local implementation of the LA model. To implement an LA program, institutions hire undergraduate students to assist instructors using student-centered, evidence-based instructional strategies in their courses. In addition to working directly with students and instructors in the classroom, LAs attend a weekly pedagogy course and a weekly preparatory meeting with the course instructors in order to help develop the necessary pedagogical and content knowledge [17].
Research has associated LA programs with lower drop, fail, and withdrawal rates [10,18,19] and improved conceptual learning [9]. These findings indicate that LA programs may have long term positive impacts on institutions' finances through increased retention. As we know from the work of the Learning Assistant Alliance (LAA) [20], LA programs require resources to start and run: local expertise, stipends for LAs, and faculty and staff time to manage the program and teach the pedagogy course. If the institutions that lack the financial resources and local expertise necessary to start and sustain an LA program also disproportionately serve students from minoritized groups, then the LA program, like any other instructional strategy, may perpetuate inequities at a national level. Our purpose in this research was to investigate the characteristics of institutions with LAs and to inform how those characteristics differ for minority serving institutions (MSIs).

II. RESEARCH QUESTIONS
To investigate equity in which types of institutions have LA programs we investigated the following research questions: • To what extent do higher education institutions with  and without LA programs differ across several institutional characteristics?  • To what extent do the characteristics of institutions with LA programs differ between MSIs and non-MSIs? These institutional characteristics included categorical variables for (1) four-year versus two-year, (2) public versus private, and (3) teaching-versus research-intensive, and continuous variables for (4) undergraduate enrollment and (5) 25th percentile ACT score, as designated by the Carnegie Classification of Institutions of Higher Education (CCIHE) [21].
Understanding the distribution of the LA model across institution types can assist the LA Alliance in either changing or maintaining the dissemination of the model to support equity broadly. While the results of the study were specific to the LA model, similar research can inform how other evidence-based reform efforts are spread across institution types. This work can provide an impetus and framework for the disciplinebased education research community to investigate how the dissemination of evidence-based strategies contributes to equity at a national level.

III. METHODS
Our data on LA programs came from the LAA's institutional database [20], the Learning About STEM Student Outcomes (LASSO) database (also [20]), and a list of Physics Teacher Education Coalition (PhysTEC) supported sites [22], noting that many PhysTEC grants include funding for LA programs. These data sources included information as to whether or not the institutions listed had LA programs; however some of this information was either incomplete or outdated. When considering those institutions for which the information was unclear, we looked at the institution's website to determine whether or not that specific institution had an LA program. We combined the list of institutions with LA programs with the CCIHE 2018 public data distribution [21].
For the first step in our analysis, we looked at the distribution of LA programs grouped by MSI status, as categorized in the CCIHE [23], and the distribution of LA programs grouped by each of the five institutional variables without accounting for MSI status. This analysis provided a baseline for identifying variables with meaningful differences. The second step in the analysis looked at the distribution of LA programs across the five institutional variables while accounting for MSI status. The five institutional variables were 25th percentile composite ACT score, undergraduate enrollment, level of institution (four-year versus two-year), 2018 BASIC Classification (teaching-intensive versus research-intensive), and control of the institution (public versus private). We aggregated research institutions using BASIC 2018 code to classify all programs granting greater than 50 PhDs per year as researchintensive, and all other institutions as teaching-intensive. Because no private for-profit institutions had LA programs, we omitted all private for-profit institutions from the analysis to avoid skewing the results.
Following these two sets of analyses, we looked into which set of institutional variables was most highly represented at all institutions with LA programs, and specifically at MSIs and non-MSIs with LA programs [24]. This analysis informed institutional patterns in the dissemination of the LA Model related to the equitable distribution of the model.
To conduct the statistical analyses, we used R-Studio [25]. We analyzed the continuous variables (ACT score and undergraduate enrollment) using means, standard errors, and Hedges' g. Hedges' g is similar to Cohen's d in standardizing the differences in terms of standard deviations, but also includes a correction for small sample sizes (N < 20) [26]. To analyze the categorical variables we used proportions, standard errors, and odds ratios as measures of effect sizes. We determined the error in our measurements using the standard error of proportions, SE = p(1 − p)/n, where p represents the proportion and n the sample size. The odds ratio is the ratio between two probabilities, such as the probability of finding an LA program at a research-intensive institution and the probability of finding an LA program at a teaching-intensive institution. The further the odds ratio is from 1 the larger the relative difference. We interpreted the size of the difference using contextual comparisons or effect size rules of thumb to inform how meaningful the differences were. We used the standard errors of the measurements to inform how confident we should be in those differences [27][28][29].
The study had several limitations. The study focused on LA programs and did not account for programs that offered similar services or the use of other evidence-based instructional strategies at institutions without LA programs. Our analysis only focused on broad characteristics of LA programs. Community colleges and private schools with smaller class sizes may see less value in implementing an LA program. Our data collection likely missed some LA programs because they were not included in any of the various data sources we used. The numbers of two-year colleges and MSIs with LAs were small (7 and 21 respectively); as such, any LA programs at MSIs that we missed could have changed our findings about the institutional characteristics of MSIs with LAs. Finally the study did not look at trends over time nor did it account for the size of the individual LA programs.

A. Overall Distribution of LA Programs
The analysis first looked at the total proportion of institutions with LA programs and the total proportion of MSIs with LA programs. Of the 3403 institutions included in our analysis, 107 institutions (3.1%) had LA programs. The distribution of LA programs at MSIs did not differ from the overall distribution across all types of institutions: 3.2% of MSIs and 3.1% of non-MSIs have LA programs as shown in Fig. 1.
We then focused on the relationships between LA pro-   Table I. Be-cause these differences were much larger than their associated standard errors, we are very confident that these results represent real differences. The odds ratios all indicated that these differences were very large for each of the categorical variables: e.g. institutions with LA programs are 14.6 times more likely to be research-intensive than teaching-intensive, 4.9 times more likely to be public than private, and 5.8 times more likely to be four-year than two-year. Of the four-year, research-intensive, public institutions in the United States, 27.1% have LA programs, contrasted with 3.1% of institutions overall. Our analysis then focused on the relationships between LA programs and institutions for continuous variables: undergraduate enrollment and mean 25th percentile ACT scores. LA programs were mainly present at larger institutions. The average undergraduate enrollment at institutions without LAs is 4676, contrasted with 17982 at institutions with LAs (Hedges' g of 1.9). Given the size of the difference and the sample size, we confidently concluded this was a large and meaningful difference. Institutions with LA programs had higher mean 25th percentile ACT scores than institutions without LA programs (22.0 vs 20.5, with a Hedges' g of 0.4, which is a medium sized difference). This difference is much larger than the standard errors: 0.38 and 0.11, respectively. This difference across institutions with and without LA programs was likely related to the difference between 25th percentile ACT scores at research-and teaching-intensive institutions. Research-intensive institutions in general have a mean 25th percentile ACT score of 22.7 contrasted with 19.8 at teaching-intensive institutions, and this difference has a corresponding Hedges' g of 0.85.

B. Distribution of LA Programs Grouped by MSI Status
We next analyzed the distribution of LA programs grouped by MSI status. We first examined the differences in the relationships between LA programs and the categorical institutional variables considered above at MSIs and non-MSIs. As shown in Fig. 2 and Table II, there is a clear difference in the distributions of LA programs at (1) two-year, (2) private, and (3) teaching-intensive institutions when grouped by MSI status. First, 5 of the 7 LA programs at two-year institutions were at MSIs. Next, no LA programs were located at private MSIs. Finally, LA programs at MSIs were twice as likely (odds ratio = 2.0) to be at teaching-intensive institutions than LA programs at non-MSIs. The total number of institutions in these three categories was small, meaning that these results could change with a small shift in the number of institutions.
We next compared the relationship between the continuous institutional variables and the distribution of LA programs at MSIs and non-MSIs. As shown in Fig. 3, the mean 25th percentile ACT scores were highest at non-MSIs with LAs and lowest at MSIs without LAs. MSIs had a mean 25th percentile ACT score of 17.7 at institutions without LAs and 19.7 at institutions with LAs. In contrast, non-MSIs had a  mean 25th percentile ACT score of 20.9 at institutions without LAs and 22.5 at institutions with LAs. The ACT score distributions at institutions with LAs are higher than those at institutions without LAs, but the ACT score distributions at all MSIs were lower than those at all non-MSIs. Undergraduate enrollment was similar at MSIs and non-MSIs with LA programs. While there is a significant difference in mean undergraduate enrollment between institutions with LAs and without LAs, the undergraduate enrollment distributions for MSIs and non-MSIs were the same when grouped by LAs.

V. DISCUSSION AND CONCLUSION
While many researchers have investigated implementations of the LA model at single institutions, we are not aware of any research regarding the distribution of LA programs. Our research indicates that the majority of LA programs are presently at four-year, research-intensive, public institutions and that the percentage distribution of LA programs is the same across MSIs and non-MSIs. However, the typical MSIs with LAs differ from the typical non-MSIs with LAs. All MSIs with LA programs are public institutions and a larger proportion of these are teaching-intensive and two-year institutions. Also, the mean 25th percentile ACT scores at MSIs with LAs are lower than those at non-MSIs with LAs.
Taken altogether these results show that LA programs are disproportionately found at certain types of institutions. There are several issues of equity tied to this disproportionate distribution, most notably in relation to differences in students' preparation and access to the types of institutions with LAs. These institutions tend to have higher 25th percentile ACT scores, suggesting that these institutions admit students who are better prepared. Subsequently, the students with access to LAs are likely to have had previous access to more academic resources, thus perpetuating inequities. Furthermore, the ACT may be biased against students of color, meaning that the higher selectivity of institutions with LAs could disproportionately impact minoritized students. Extending the benefits of the LA model beyond this select group of institutions to more two-year and teaching-intensive institutions will more equitably distribute these benefits nationally.
Institutions with LAs are predominantly four-year, which historically underserve students from minoritized and socioeconomically disadvantaged groups. Many students begin their undergraduate education at two-year institutions for economic reasons. Two-year institutions tend to have fewer academic resources and lower graduation rates than four-year institutions [30]. LA programs are associated with lower drop, fail, and withdrawal rates [10,18,19], meaning that they could have a large impact on students at two-year colleges. It is common for students to transfer into four-year institutions after completing their entry-level courses at two-year institutions. LA programs are mainly implemented in introductory courses, so students who do not begin their degree at a four-year institution are unlikely to have LAs in their courses, even if they graduate from a university with an LA program. Furthermore, those students who transfer into a four-year institution with an LA program could miss the opportunity to become an LA, since they compete for LA positions against students who experienced LA supported instruction.
We have attributed the disproportionate distribution of LA programs at four-year institutions to issues of equity; however, there may be fewer LA programs at two-year institutions because these institutions already have smaller class sizes and other structures in place to support active learning and student success. We would also expect to find LA programs at larger institutions because these institutions tend to have larger class sizes, especially in introductory STEM courses. LA programs allow the instructors in these courses to implement evidence-based instructional strategies that require low student to teacher ratios and therefore can have a greater impact at larger institutions.
Finally, we discuss the origins and dissemination of the LA model [16]. Discipline based education researchers developed the LA model at a large, four-year, research-intensive, public, non-MSI. Their LA program addressed the need to reduce student to teacher ratios to allow using student-centered, evidence-based instructional strategies in large-enrollment physics courses. At the same time, those large-enrollment courses provided large samples for quantitative research that reinforced the value of the pedagogical transformations. Being at a research-intensive institution provided this initial LA program with DBER expertise to implement the LA model and positioned the program to seek and gain external grant funding to support the fledgling model. The LA model has predominantly expanded to similar institutions with similar resources, needs, and constraints. However, those institutions predominantly serve middle-and upper-class white students. We are encouraged by our findings of the equal proportion of MSIs with LAs and the small but likely growing number of two-year colleges with LAs, indicating that the LA model is expanding to serve more students from minoritized groups. These findings can act as a benchmark to track who has access to the LA model and as a guideline for similar investigations on other evidence-based instructional strategies.