Physics GRE Requirements Create Uneven Playing Field for Graduate Applicants

The use of the Physics GRE in graduate admissions has gained considerable attention in recent years. While studies have shown the problematic nature of the exam quantitatively, it is time that student experiences were also included in the discussion. In this qualitative study, we interviewed 69 current graduate physics and astronomy students about their process of deciding where to apply to graduate school. Physics GRE requirements played a substantial role in this decision, with 48 students mentioning the test as part of their process to narrow programs down to a personalized short-list. Participants discussed potential barriers (e.g., financial and travel) that affected some students, but not others, which created an unequal playing field for grad school applicants. Secondly, while the participants noticed a shift towards more departments having “optional” GRE language, female student participants still felt the need to take and submit their Physics GRE scores, while male student participants truly saw “optional” as optional. These results suggest that graduate programs requiring the Physics GRE are doing more than asking prospective graduate students to simply take a physics test, and that the use of ‘optional’ requirement language may be inadvertently disadvantaging the very students that they are trying to recruit.


I. INTRODUCTION
The strive towards holistic admissions within physics graduate programs has been a growing movement in recent years; yet, some graduate programs continue to rely heavily on quantitative measures such as an applicant's Grade Point average (GPA) and Physics Grade Record Exam (GRE) scores and neglect other factors when making graduate admissions decisions [1? -4]. Studies have shown the problematic use of Physics GRE scores due to a performance bias against women and underrepresented minorities [6][7][8]. At the same time, incorporating holistic review for STEM graduate admissions allows for such quantitative measures to still be utilized, while reducing risk of racial/ethnic and gender bias by having GRE scores (general and subject-test) as just one element within a broad holistic graduate application [9,10]. However, much of the literature on the use of Physics GREs in graduate admissions has been focused exclusively on the faculty and programmatic side of this equation with student voices remaining largely absent. In order to understand the full impact of equity issues regarding the Physics GRE, we believe that it is necessary to also elevate the voices of students who are responsible for preparing for, paying for, and taking this exam as part of their graduate school application.
In this paper we will discuss the views of 69 current physics and astronomy graduate students from 27 different Ph.D. institutions regarding their experiences with the Physics GRE as part of their graduate school application process. First, an emergent theme of barriers (financial and travel) students faced when taking the Physics GRE will be discussed. Then, we will show that 'optional' Physics GRE requirements elicited different reactions from male and female students. Implications for graduate school recruitment and admissions will be also be discussed in light of this work.

II. BACKGROUND
Logistical and financial considerations of the Physics GRE emerged in the findings as influential on students' application process. To provide context for those results, this section outlines some background information about the structure of the test according to the Educational Testing Service (ETS) website for test-takers in the United States [12]. For international students, the costs, available test dates, and number of testing locations varied by country, with many countries having fewer than a dozen sites total; for example China (13), India (7), Russia (4), Israel (1), and Mexico (2).
As a paper-based test, the Physics GRE is offered three times of the year (September, October, and April) in the US. Scores are mailed a month following the exam. The cost of the Physics GRE is $150, and students are permitted to send their scores to four institutions, which must be named at the time of registration (before the student has taken the test). Each additional mailing of scores incurs an additional $27 dollar fee. For students also required to take the General GRE, the cost is an additional $160 dollars, and follows the same procedure and fee schedule for sending scores.
Unlike the General GRE, which is often computer-based and can be administered at thousands of locations, paperbased tests are only available at 225 select testing centers, with five states (AZ, NV, NH, ND, and VT) offering a single testing center in their state. Further, not every paper-based testing center offers the Physics GRE three times each year.

III. METHODS
Between 2018 and 2020, the first author traveled to five different Ph.D. programs as well as three international physics conferences (such as the APS March Meeting) and conducted student focus-groups as part of a larger project focused on holistic graduate admissions and retention practices. At three program, the first author sent a prescribed email to graduate students inviting students to participate in focus groups. The groups varied in size from one on one sessions to as large as six people per group based on student availability and comfort being interviewed in front of their peers. The student participants (N = 69) were from 27 different graduate programs in physics or astronomy. Table I shows a demographic breakdown of student participants. Students self-selected all demographic information. Focus groups were advertised as 60minutes in length, but some groups wanted to continue their discussion and ran as long as 90-minutes. All focus groups were audio-recorded and transcribed via a secured third party service.
One of the admissions-related questions asked to students was, "How did you decide where to apply to graduate school?" There were two goals in asking this question. The first was to understand what factors weighed into students' application decisions; second, to determine the relative weight of each factor. However, in the midst of the interviews, the first author began to notice that students were especially vocal about the Physics GRE, specifically in terms of barriers they had to overcome (or in some cases, could not overcome) in order to take the exam. In response to participants' discussion, students were encouraged to share their experiences about the nature of financial and travel barriers they faced (if any). This allowed the first author to better understand students' experiences and tease out the circumstances that affected some students but not others.
Later in the focus-groups, graduate student participants were also asked "Where did you apply to graduate school? Why those programs?" The goal of this question-pair was to provide additional information for students' decision making process in selecting potential graduate programs. The Physics GRE was once more repeatedly mentioned as part of student responses, especially when students discussed applying to Physics GRE 'optional' programs. In response to students' dialogue, the first author asked a follow-up question about students' justification for sending or not sending their Physics GRE scores to programs with 'optional' requirements to discern if a pattern among students was emergent.
Transcripts were analyzed and coded in NVivo to identify barriers students discussed in reflecting on their preparation for, taking, or following the Physics GRE. In addition students' discussion of 'optional' Physics GRE requirements were coded to identify any relationships between 'optional' scores and decisions about where to apply. In these cases, we use 'optional' to encompass multiple requirement descriptions including: 'not required,' 'optional,' 'recommended, but no minimum required score,' and others. A colleague, not affiliated with this project, coded the "admissions" section of one of the graduate student focus group transcripts. This second rater was provided a code book by the first author, and comparison of codes results in over 90% agreement.

IV. RESULTS
Overall, students listed a variety of factors that helped them narrow the nearly 184 potential physics Ph.D. programs and 51 astronomy Ph.D. programs down to their personal shortlist, with the Physics GRE being highly influential. This section will first quickly discuss a group of students for which the Physics GRE was inconsequential. Then, we will present two barriers students' discussed when reflecting on the test, specifically financial and travel. Next, we share results from students who were not able to overcome those barriers and were unable to take the Physics GRE at all. Finally, we will present student views on their interpretation of "optional" Physics GRE requirements.

A. Physics GRE: Barriers
The graduate students' discussion of the Physics GRE was often centered around associated barriers with taking the exam. For example, the financial and travel costs associated with the exam were insignificant for some and insurmountable for others. Those students who did not take the Physics GRE were further limited in their choice of graduate programs, and could only apply to programs that either did not require the exam or had "optional" language.

No Barriers Discussed
In thinking about the process of applying to graduate school, 19 students (11-F / 8-M) did not mention the Physics GRE as part of their decision making process. A closer look at this group revealed that nine of these students attended "elite" graduate programs (i.e., Top 10 programs based on their program's National Research Council Ranking), many of whom had multiple acceptance offers from such highly ranked programs. Only one student self-identified as an underrepresented minority in this group; they did not mention the Physics GRE as part of their application process because the APS Bridge Program to which they initially applied (before transitioning to a Ph.D.) did not require it, and therefore they never took the exam.
Another question later in the interview protocol asked students to describe their experiences taking the Physics GRE. It was revealed at that time that six students who did not discuss barriers had not taken the test, and had only applied (and were therefore enrolled) in Ph.D. programs that either did not require the Physics GRE or had "optional" GRE language. Those who had taken it (N = 13) were nonchalant towards the exam, often claiming that they scored very well (70th percentile or higher in all but one case) and saw Physics GRE as simply an inconvenience.

Financial Barriers
Within the interviews, students were especially vocal about the cost the Physics GRE in terms of applying to graduate schools. While some students appeared to not be affected by the cost of graduate applications, nine students specifically cited the total monetary cost of the Physics GRE as being outside of their financial means and relied on credit cards or loans to cover graduate admissions related expenses. While the number of graduate programs the participants applied to varied widely, many were advised to apply to a range of 8-12. One student cited being advised to apply to at least 10 programs, and determined that it would require a budget of around $1500, which for many participants was out of their financial means. To overcome financial limitations, one student described their frustration over advice to accrue more debt as part of the application process: Actually, the advice I was given was get a credit card, which is not really something who's five figures in debt wants to hear. It's like, "Yeah, just pile up more debt," [...] So I'm going to have that for at least a few months sitting there on that card gaining interest. This is not something that I appreciate.
Students opting to take the exam in September described wanting to have a second chance in October, just in case their initial scores were not as high as they had hoped. However, not everyone was financially able to afford take the exam a second time. For example, I did very badly my first time. And I guess that was another reason why I was struggling to get myself to apply to other schools was because I did really bad and I wanted to retake it, but I couldn't afford it. It's almost $200. And I mean, I wasn't working because I was senior year of undergrad and I was just taking out a lot of loans to get through. And I was like, I can't do this. And so, I had to just roll with the really bad score that I had. This student was enrolled in a program that did not require the Physics GRE, and was the only program to which she was accepted.
Additional financial burdens were placed on students who had to travel outside of their city in order to take the Physics GRE, which ties into the second theme of travel barriers.

Travel Barriers
All students had to take the necessary preparations to be ready for the 8:00am (local time) start. However, many students, especially those in more rural undergraduate programs, cited having to travel up to 300 miles in order to get to the testing location. In addition to gas (or public transportation) money, these students also needed to pay for a hotel room the night before the exam, incurring additional associated costs.
Having to travel to take the Physics GRE without ownership of a car presented an even bigger challenge for some students. For international students whose own country did not offer the Physics GRE, visiting a neighboring country in order to take the exam was required. One international student from the UK described how he traveled to the testing location by bus: I had to go across half the country on a bus to take the GRE. And again taking the Physics GRE, the difficulty there was the only test location was [four and a half hours away].
Many universities now have branch campuses spread across a state. One student did not realize that even though his university was listed as a testing site, it was not actually offered at his campus.
So the Physics GRE-I'm new to [this Southwest city]. And, in fact, come to find out that the Physics GRE was done at the local college. But it wasn't on their [main campus] branch, it was like the branch 300 miles west of that. Like in a smaller town, but it was the same college name. And I didn't know any better, and so I didn't realize that until the morning of [my exam].

Did Not Take Physics GRE
A third of the graduate student participants (11-F / 11-M) never took the Physics GRE (four of these students (3-F / 1-M) also self-identified as a traditionally underrepresented minority in physics). Lack of financial means was by far the biggest deterrent for this group with many claiming that they could not afford the total cost of the exam. Students in this group often only applied to 2-3 graduate programs (all of which had no formal Physics GRE requirements) and were most likely to take advantage of any potential graduate application fee waivers that programs advertised. Two international students also fell into this category due to problematic travel considerations.

B. Physics GRE: "Optional" Requirements
There were 19 graduate student participants (10-F / 9-M) that took the Physics GRE and had applied to at least one program that listed the Physics GRE as "optional". As shown in Table II, when students were asked whether or not they sent their Physics GRE scores to those programs with 'optional' requirements, all 10 of the female students cited that they had. Conversely, half of the male students in this group sent their scores, claiming to have scored well, while the other half cited a poor score as their justification for not including their Physics GRE scores on applications destined for program with 'optional' requirements. Male students also did not seem concerned with hiding their scores, expressing that if their score would potentially hurt their application, they relied on that 'optional' language to justify not sending their Physics GRE scores. When female graduate students were asked why they sent their (self-proclaimed) low scores to every program, even when it was optional one student cited that she was advised to send her scores saying, "I didn't want them to think that I had something to hide". Another student claimed, Unless they said, 'Do not send,' which [my program] did, I sent them anyway because everyone online was like, 'You should send it, just so they don't think you got a zero.' These two sentiments rang true for all of the female students except one, who was genuinely proud of her high score and felt that including it may strengthen her application. Similarly, if a male student felt that their score was high, or that it would help strengthen their application, they included it.

V. DISCUSSION AND CONCLUSION
The goal of this study was to understand how Physics GRE requirements influence applicants' decisions and ability to apply to graduate programs in physics.
The total monetary cost of the Physics GRE (e.g., exam, sending scores, and travel) was an inconvenience for some, and insurmountable for others. Further, those with additional financial resources were inherently able to apply to more graduate programs and could afford to travel or take the Physics GRE multiple times, if desired. However, other students could only afford to take the test once or not at all. Therefore, asking students to complete the Physics GRE as part of their requirements for graduate school can inadvertently disadvantage students whose financial situation limits the number of times they can take the Physics GRE or the number of scores that they can afford to send. In addition, while some states (CA, PA, NY) offer over 10 testing sites in their states, 28 states offer four (or fewer) testing locations. Students having to travel to take their exam, including some international student applicants, incur additional financial and logistical burden over their peers.
For female students, submitting no score was seen as more suspicious than submitting a bad score, explaining why nine of ten submitted their Physics GRE scores to programs that listed 'optional' requirements. For their male peers, poor performance on the exam was cited as the justification for not sending their scores, claiming that no score would be less detrimental than a bad score (the opposite view of the female students).

A. Implications and Limitations
There are two implications of this work. The first is that graduate programs requiring the Physics GRE are doing more than asking students to take a physics test. While the cost of taking the exam and sending scores was equal for all test takers in the US, the relative financial impact on students was not. The associated cost of the Physics GRE ranged from not being a concern through financially straining to financially out of reach. Being aware of the financial and travel stresses that the Physics GRE imposes on certain students can help faculty serving on physics graduate admissions programs understand the additional 'asks' they are placing on some applicants but not others by requiring the Physics GRE for graduate school admissions.
Secondly, while some graduate programs are transitioning to the use of 'optional' Physics GRE requirements, female students interpreted this language as 'required' while male students truly saw it as 'optional'. Also, because female students felt more pressure to send their Physics GRE scores to 'optional' programs, they may also feel increased pressure to take the exam, even in the face of financial and travel barriers. To combat the discrepancies seen across gender, we suggest that programs avoid optional language and instead clearly state 'Do Not Send' on their admissions website. In addition, advertising that the Physics GRE is not required opens the door for the subpopulation of applicants who never took the Physics GRE (33% of our participants) to apply.
There are two notable limitations of this study. First, the 69 participants in this study do not represent the views and experiences of physics graduate student population as a whole. Further, since all participants were already enrolled in graduate school, this current study failed to capture the views of those who applied to graduate school, but did not attend. Future work could include a quantitative survey given in the Spring of senior year to undergraduate students, asking about how the Physics GRE is affecting their graduate application decisions, potential barriers to taking the test, and if they included Physics GRE scores in their applications to 'testoptional' programs. These results would allow for a more generalizable findings on the impact of Physics GRE use.