Believe that they can achieve: How Teacher Attitudes Toward Physics Impact Student Outcomes

To address the longstanding problem of underrepresentation of women in physics, we developed two classroom interventions that encourage womens’ future physics intentions. In testing these lessons in a larger study, we found variance in gains between student sub-populations across several teachers. This prompted the current mixed methods analysis to follow up on potential contextual factors leading to these differences, including social and economic setting of the school and student population characteristics, as well as teacher-level effects. We drew upon multiple sources of data collected from both teachers and students including teacher interviews, teacher and student open response surveys, and student artifacts from the lessons. In our preliminary analysis, we found that the broader social and economic environments did not appear to affect how students received the lessons; however, individual teacher implementation of the lessons did.


I. INTRODUCTION
Addressing the longstanding issue of underrepresentation of women in physics, we developed two classroom lessons based in equity and gender literature recommendations and in close collaboration between a team of researchers and high school physics teachers. These lessons expose students to different ideas about what physics is and who physicists are, which helps students from traditionally underrepresented groups in physics to see themselves in a physicsrelated career. These lessons were designed to encourage the physics identities of physics students in high school, particularly female-identifying students and encourage them to consider careers in physics, with the ultimate goal of increasing the representation of women in physics. These interventions were experimentally tested in Fall 2018 and were found to have a positive impact on students, importantly females and underrepresented minorities [1,2], however, there were variances between teachers in which student sub-populations had the greatest gains.
In the current case study we sought to follow up with the different cases, such as identifying how local factors such as school socio-economic contexts, teacher background, or classroom environments were associated with the studentlevel outcomes of experiencing these lessons. In addition to these factors, we later questioned how teacher beliefs could affect how students benefit from these lessons. It has been shown that teacher beliefs on what is required to be successful in a particular subject area can have a large impact on students' self efficacy and performance [3][4][5][6]. This extends to teacher's beliefs on multicultural issues as well [7]. Teachers who unconsciously communicate these beliefs to their students may dissuade their students from pursuing that particular field further.

II. METHODOLOGY
This study follows a mixed methods design, first identifying cases based upon analysis of quantitative student survey data, and investigating the cases qualitatively for deeper insight about classroom contextual factors, teacher implementation, and student experiences. Our guiding questions are: What similarities and differences exist between teacher/classroom cases with varying student outcomes? Does local context (school, state, local socioeconomic status) impact student outcomes?

A. Data Collection
Student data: The students of the five participating teachers in this study were surveyed at six points during the semester: once early in the semester, before and after both classroom lessons, and again near the end of the semester. Amongst the outcomes measured in these surveys, we were most interested in the gains in future physics intentions of the students. We also collected student essays from the lessons. Teacher data: Of the 28 teachers in the main study, five had been interviewed at the time of analysis. Teachers were interviewed at the end of the semester about their experiences teaching the lessons, professional backgrounds, relationship to physics, their school and classroom environments, and how they felt students reacted to the lessons. In addition, teachers filled out open-response surveys about their implementation of the lessons and provided feedback for improving them.

B. Analysis
A starting point for this work was a preliminary analysis of the gains in the future physics intentions of the students following their experience with the lessons, which led us to begin formulating questions about each teacher and their implementation of the materials. We also extracted information from the survey data to better describe each teachers' classroom. This included data regarding gender breakdown, student career interests, and student goal endorsement. Nonparametric wilcoxon rank-sum tests were used to determine if each class differed, on average, from the larger sample of students in regards to future physics intentions gains, home support and grades, detailed in Tables I and II. To follow up on initial questions emerging from the quantitative analysis, we looked into the qualitative data to build cases, the unit of analysis being the teacher-classroom cases using the teacher interview and student artifact data. We used explanation building, comparing initial propositions emerging from one case with the other cases and revising our propositions until reaching a satisfactory explanation [8]. This analysis was discussed as a group and primarily led by one researcher. In examining teacher interviews, we looked for evidence that could inform us about the teachers' attitudes toward physics, as well as any adaptations the teachers may have made to the lessons (to account for their classroom contexts). Conclusions were triangulated by continually checking against other data sources such as the teacher surveys and student artifacts.

III. RESULTS AND DISCUSSION
As this is a work in progress, the results presented here are preliminary and based on the analysis of the data discussed above. We present an overview of each teacher, their classroom contexts, and student populations in Table  I. An overview of student responses to pre-survey questions are given in Table II. We have binned teachers into one of three groups, determined by what students saw gains over the semester. Mr. Apple and Ms. Bignay both saw gains for only their non-female students. Ms. Cherry saw gains for only her female students and Ms. Date and Mr. Elderberry saw gains for all of their students. TABLE I. Overview of five teachers and school contexts. Gains refer to significant gains in future physics intentions (F -Female, NF -Non-female), measured between round 2 and 6 of surveys. Years teaching refers to the number of years teaching physics specifically. SES bins have been determined by the percentage of students receiving free and reduced lunch (FRL). High SES refers to schools with a low number of students receiving FRL. ***: p ≤ 0.001, **: p ≤ 0.01, *: p ≤ 0.05 Both Mr. Apple and Ms. Bignay experienced future physics intention gains for only their non-female identified students. Both teachers are also relatively new to teaching physics, each having only taught the course for one year at the time of the study. The similarities end there, however. From Table I we see that the classrooms of these two teachers are quite different, Mr. Apple teaching in a low SES rural school in Texas and Ms. Bignay teaching in a high SES school in urban Virginia. Mr. Apple taught two sections of regular physics while Ms. Bignay taught one section of AP physics and two sections of regular physics. In addition to these differences, Ms. Bignay teaches physics at a school that focuses primarily on visual and performing arts.
We sought to understand why only the non-female students were impacted by the lessons in these two classes, considering the lessons were designed to be most impactful for female students. First, we examined student-level data collected from pre-surveys administered before lessons took place. We found Mr. Apple's students to be representative of the study in most areas while having higher than average grades in math and English. It seems unlikely that students' higher grades in these areas would lead to female students benefiting less from the lessons so an analysis of Mr. Apple's interview followed. In his interview, Mr. Apple comments on his experience with physics majors while in college: Yes, actually was really surprised by the statistics . . . my buddies that were physics majors [were] just brilliantly smart, but I never really considered it.
Mr. Apple appears to espouse the idea that physicists are in general extremely intelligent, possibly to the point where it almost seems unachievable. This is a common myth held by many people, and importantly students [9]. This kind of association with brilliance has been shown to drive underrepresented groups away from fields associated with innate brilliance [10]. Had Mr. Apple been reinforcing this standard belief about physicists in class, the lessons may have only served to benefit students who already identify with the field and not allow new students to feel welcome or able.
As stated before, Ms. Bignay teaches at a high school whose students focus primarily on the arts. This offered a unique perspective on how students with very focused motivations interact with these lessons and how able they are to connect the benefits of physics to their desired career paths. As with Mr. Apple, Ms. Bignay's non-female students were the only group to see significant gains in their future physics intentions. Once again we wanted to understand if there were any factors about the school environment that may have affected how these students perceived the lesson, especially since this school was so unique from any other school in the study. In this case, students may have had a harder time finding connections between physics and the arts. The lesson on careers in physics had several physicist profiles detailing individuals who use their physics degrees in art-related careers, however, this may not have been effective. Based on details found in Ms. Bignay's interview, the teacher may have had a difficult time connecting the concepts herself. When dis-cussing how she prepared for the lesson, she states: . . . my immediate thought was this isn't going to work for my students because most of my students are intending to go into something arts related . . . When describing how the lessons went, Ms. Bignay mentions how many of her students had difficulty finding a way to use a physics degree in the career they were interested in. Ms. Bignay recounts how she responded to some of her students: I said 'All right. Just for fun, think about what the writers of this lesson were trying to do and just think outside of the box. It doesn't have to be something that you necessarily would do in real life' Ms. Bignay goes on to explain how her students ended up being very creative with their responses to the lesson, however the implementation of this portion of the lesson may not have benefited the students as much since the students were not challenged as much to connect physics to their own lives.
While the lessons were designed around generating meaningful class discussions and changing student perceptions about physics, these two teachers may have not challenged these stereotypes enough, and in some instances believed the stereotypes themselves.

Ms. Cherry
Ms. Cherry is an experienced teacher in suburban Virginia who has been teaching high school for over forty years, over ten in physics. During the study Ms. Cherry taught two sections of regular physics and one AP course. In direct contrast to the results of Mr. Apple and Ms. Bignay's classes, Ms. Cherry saw gains for only her female students during the semester. We again looked to see how Ms. Cherry's students compared to our larger sample and found that her students were also similar with the exception of math and English grades. In her interview, Ms. Cherry stated that her students were very engaged with the lessons. A unique aspect of this class was that the teacher would play music during small group discussion: I have a playlist of the happiest songs ever. I play music just loud enough so that people in their group can hear each other, but they're not going to be listening to other people across the room.
She expands on this statement later, saying: . . . the music just helps because they feel a little bit more comfortable with people [that] aren't listening in on them. Especially me, I'm not listening in on them.
She also allowed students to sit with whomever they pleased, stating that "They're already sitting with friends that they feel comfortable with" so that more productive discussions can be had. It appears then that Ms. Cherry takes extra care to ensure that her students feel safe and comfortable during class.
One of the cornerstones of the lessons is an interactive class discussion of each topic, specifically the lesson on underrepresentation in physics. Ms. Cherry also appears to be very aware of issues regarding gender in the modern world, stating: ...I had to outright tell [the students] that the equal rights amendment is still trying to be ratified. That they really thought that it was in the constitution that women have equal rights.
The emphasis on these particular beliefs may have been more beneficial for the female students in Ms. Cherry's class than the non-female students. Ms. Cherry also appears to espouse the idea of a growth mindset [11], at one point saying "I want them to be comfortable, but I want them to also feel capable." Having a growth mindset, the idea that knowledge is something that is developed rather than innate, is a very powerful method for engaging students and making them feel welcome in a class or field. The beliefs Ms. Cherry has in regards to gender issues and growth mindsets could explain why only the female students in her class saw significant gains in their future physics intentions.

Ms. Date & Mr. Elderberry
Ms. Date teaches physics in urban Texas, and Mr. Elderberry teaches in an urban Florida school. Both have over ten years of high school physics teaching experience. From the tables above we see that, while the gains for students are similar, the class environments of these two teachers was quite different. Ms. Date teaches physics primarily to at-risk students with learning disabilities in a low income area, while Mr. Elderberry teaches to students in a medium SES magnet school. Additionally, students receive varying degrees of home support toward physics. Ms. Date's students are not typically pushed towards physics while Mr. Elderberry's students are highly encourage to do so by their families. Despite these differences, both teachers saw gains for all students, female and non-female.
From the interviews it is clear that both Ms. Date and Mr. Elderberry are very enthusiastic about physics. Because she teaches in an at-risk teaching environment, Ms. Date must take extra steps to ensure that her students feel welcome and engaged. This may have led her to tailor the lessons to be more beneficial to her students. Ms. Date comments on her ability to structure her class to her students: [The school] gave me lots of leverage in how I would be able to structure my class and do a lot of project based planning and have them be successful.
Ms. Date puts in a very serious effort to encourage her students to succeed. When discussing her school, she states: It's a traditional public school. Very high needs area, low economics, poor performing, very, very challenged students. That's why it's so awesome for me to be able to do all these wonderful physics things with them, because they needed to believe that they can achieve.
Similarly, Mr. Elderberry took extra steps to ensure that his students, particularly female students, were encouraged and acknowledged during lessons: . . . [teaching is] a little bit of an art form, kind of, where you kind of see it like the ones that are struggling or they're too shy to answer, how do you get them involved.
It is also important to note that Mr. Elderberry teaches a physics modeling class rather than a traditional lecture based course. Students are often tasked with solving and explaining problems to their peers in open discussion meetings.
I think a lot of it was modeling and having the students go up and explain the work, and a lot of the girls got comfortable with presenting work, and they would be able to explain it. And then it built their confidence.
In the case of Mr. Elderberry's students, they may have been more prepared to engage in the discourse associated with the two lessons, allowing all students to see gains in their physics intentions.
Both Ms. Date and Mr. Elderberry talk about the lessons and their teaching in terms of a growth mindset. These two teachers challenge their students to think critically about the usefulness of a physics degree to their professional careers, leading them to really consider the degree rather than go through the motions of a lesson. There is evidence that while growth mindset is beneficial for all students, it is particularly effective for stigmatized students [12,13]. This could explain why even though Ms. Cherry also held these kinds of beliefs, only the marginalized students in her class (females) saw significant gains. Ms. Date and Mr. Elderberry teach to predominantly black and latinx students respectively, which are both marginalized groups within physics.

IV. CONCLUSIONS
While this study is still ongoing, there is evidence for suggested implementation of the materials and avenues for future work. There appears to be a correlation between teacher experience and attitude toward physics and the level of gains experienced by the students in each class. We found that in the cases of the two relatively inexperienced teachers, female students did not have gains in their future physics intentions. There are two factors to potentially tease out in future work: level of experience teaching physics, and apparent teacher attitudes toward physics. The less experienced teachers expressed attitudes in line with the traditional thinking about physics (e.g., one must be "brilliant" to be a physicist, physics is difficult). Whether these attitudes would shift with more experience, or if the more experienced teachers in the present study entered into physics teaching with non-traditional attitudes to start (perhaps contributing to their persistence), is unclear.
A limitation when describing "local context" is the supportiveness of the school administration and local community for the teachers' decisions to implement equity-focused lessons; as these teachers were all participants in a research study, their administration was aware of and supported this research. Many teachers are unfortunately in less supportive settings, which in turn could impact how they implement the materials, if at all.
There are two key takeaways for teachers interested in implementing the materials. First, we have evidence that teacher beliefs play a larger role in student gains than student backgrounds and environments. Students from both high and low SES schools saw gains for female students. This is positive news for any teacher that may be hesitant to implement the lessons for fear that it may not impact their students positively given their local context. The characteristics of students (amount of support for taking physics at home, grades, race, career interests) also did not impact student gains in aggregate. Our team often hears that teachers don't believe these lessons will be beneficial for their students due to their students' backgrounds or interests. Based on these results, the teacher can be the motivating force for change.
The second takeaway is more cautionary: it is possible to not achieve the goal of encouraging more women to pursue physics even when implementing these carefully crafted and tested lessons. The theoretical basis behind the lessons is undermined when students receive messaging reinforcing stereotypical beliefs and norms surrounding physics, which the less experienced teachers mentioned in interviews. This is not to discourage newer teachers from implementing these particular or any equity-focused lessons, but to be mindful of reinforcing typical beliefs toward physics in the classroom. One of these beliefs, the idea that physicists are "brilliant," reinforces the idea that only certain people can do physics. Instead, what can be done is fostering a growth mindset that makes physics more accessible to all.