Mindset in context: developing new methodologies to study mindset in interview data

Mindset is a well-studied area of the psychology literature with a straightforward proposition: if you believe it’s possible to grow your intelligence, you are more likely to improve in school. The proposed mechanism behind this link is that students with growth mindset exhibit “mastery-oriented behaviors” in response to challenges, such as embracing hard work and strategizing how to improve. Mindset is typically studied through large-N survey measures with context-general Likert scale items measuring beliefs about the nature of talent and intelligence. We argue that such context-general survey measures are limited in application, particularly in the context of introductory college STEM courses. We describe a novel mindset coding scheme for interview data and apply it to a set of exploratory interviews toward examining how mindset may exhibit some context-dependence.


INTRODUCTION
Mindset is a longstanding area of psychological research centered on learning goals, beliefs about intelligence, and the consequences of holding such beliefs. Broadly, people are often referred to as having a fixed or growth mindset. Yeager, Dweck, and colleagues recently described that, "When students have a fixed mindset, they believe that their intelligence is something that is finite and unchangeable...when students have more of a growth mindset, they believe that intelligence can be developed" [1]. Interventions aimed at student mindset have had remarkable success on middle school math grades and promising results with college persistence [1,2]. Yet, there is limited work on mindset in the context of introductory college STEM experiences. In addition, our ability to understand contextual factors influencing mindset is extremely limited. This paper is focused on methodology toward this goal: we present a novel coding scheme for coding mindset in interview data and describe some preliminary analysis on exploratory interviews. We explore questions raised by our results.

A. Context Dependent Beliefs
The history of work on context-dependent epistemological beliefs in physics shows a promising path in exploring mindset beliefs. Originally, researchers treated students as having stable epistemological beliefs, for example, 'believing that knowledge comes from authority' such that learning is understood to be a process of remembering and regurgitating knowledge [3]. Later, the research demonstrated how beliefs about learning are contextually activated and students can be primed to activate particular beliefs, such as the belief that 'there is a connection between mathematical formalism and common sense' [4]. The emphasis on context-dependent beliefs constituted a breakthrough as it gave instructors strategies to frame activities to draw on students' most productive beliefs [5].
In the mindset literature, Dweck herself recently stated, "maybe we talked too much about people having one mindset or the other, rather than portraying people as mixtures," [6]. Mindset would benefit from contextdependent work similar to epistemological beliefs because it would provide instructors strategies to draw on students' most productive mindset beliefs.
In order to examine context-dependence, mindset research needs ways of collecting and analyzing data that move beyond standard context-general survey tools. The work on context-dependent beliefs in physics education has strongly benefited from fine-grained analysis of interview data (e.g. [4,7]). Currently, the work on mindset does not include any studies directly attempting to measure mindset in interview data.

B. Work on Mindset
In practice, the psychology literature approaches mindset in two main ways: beliefs about intelligence (measured through surveys) and learning orientations (measured through categorizing individual's actions in response to challenges). Beliefs are described as either "entity theory" or "incremental theory" [8,9]. Learning orientations are described as either "mastery orientation" or "performance orientation" [10]. Mindset has come to umbrella even further ideas [1], but in this paper we focus on the central topics of beliefs and learning orientations. We will use the term growth mindset interchangeably with incremental theory and performance orientation. We will use the term fixed mindset interchangeably with entity theory and mastery orientation.
Mindset has been shown to be consequential in college STEM with respect to grades, belonging, and decisions to major [11,12,13]. Studies typically take an approach to measuring entity and incremental theory either through domain-general or math-specific surveys. We take a novel approach of developing an interview coding scheme to examine mindset across context, with a focus on college physics.

DEVELOPMENT OF INTERVIEW PROTOCOL
In order to examine mindset in interview data in different contexts, we began by designing a protocol that we expected would elicit mindset-related talk. Rather than starting with context-general survey items, we instead asked students to describe and explain some specific in-school and family-related experiences. Due to space constraints we cannot share the entire protocol here, but we will highlight questions that help interpret the analysis that we present. As stated previously, there are no examples in the literature to follow for how to study context-dependent mindset. However, in designing interview questions, we drew on studies that found mindset-related talk in college computer science and high school math contexts [13,14]. We began with an exploratory interview designed to follow up conversationally in directions that seemed fruitful.
Our protocol questions focused on students' introductory physics course experiences, but also asked about other disciplines, family stories about science, students' strengths, and interactions with peers. We also asked questions that we anticipated would more often elicit fixed mindset ideas such as, "Did anything seem hard/impossible at first in physics, but eventually you were able to understand?" (growth) and "Is your family surprised that you're pursuing your path of study?" (fixed).
In this study we consider context in two ways. First, we consider the context of student story-telling that students reference when telling stories. Second, we consider the role of the interviewer and their questions in describing the context of student dialogue as questions could tip students into exhibiting different beliefs.

SUBJECT POPULATION
For the purposes of this pilot study we interviewed 5 students enrolled in an introductory calculus-based physics course for mostly life science majors at a large Midwestern research university. Our participants included 4 biology majors and 1 physics major. They predominately self-identified as female and White.

DEVELOPMENT OF CODING SCHEME
In addition to an interview protocol, we developed categories for coding mindset statements when they appeared in the interviews. We follow in the qualitative methodology tradition of progressive refinement of hypotheses [15]. We started by asking simply, "can we see evidence for mindset in interview data?" We developed categories directly from the psychology literature on mindset, and situated the categories broadly as located in "belief statements" and "learning orientation statements" that we explicate in detail, below.

A. Belief statements
Entity and Incremental Theories are typically measured through surveys that examine two wordsnamely "talent" and "intelligence," -and whether those words are believed to be fixed or changeable [9]. After finding zero examples of students using either of these words across all five interviews, we developed two additional categories: selected "synonyms" (e.g. brilliant, genius, gifted) and "nearby" belief statements (good at, bad at, suck, natural, affinity, inheriting, genetic, smart, stupid, dumb, science stuff). The latter category arose from our data as well as student quotes from other work related to mindset [13,14,16]. For all synonym and nearby words, we found every instance of them in interview transcripts, including minor linguistic variations. We then determined whether the word was used by the interviewee in a changeable or fixed way.

B. Learning orientation statements
A person's learning orientation is characterized in the literature as their response to challenge. For coding, we found student story-telling around challenges and then categorized students' responses according to Dweck and colleagues' descriptions of what constitutes mastery or performance-oriented approaches. For masteryoriented approaches, we coded whether students described effort or hard work [1], "studying to learn, not just ace the test" [16], and/or changing their strategies [9]. For performance-oriented approaches, we coded whether students described assessing their ability and giving up if it was low [9]. See Table 1:

PILOT DATA ANALYSIS
In this section we start by presenting examples of how our coding scheme played out in the data, and then present a promising example of context dependence. We caution the reader that that the analysis we present is preliminary toward articulating a more robust analytic framework for mindset and thus lacks inter-rater reliability. Nevertheless, our analysis shows that coding for mindset in interview data is sometimes possible.
In total, we found instances of 17 synonyms and 40 nearby belief words across all 5 interviews. Most promising, "good at" was strongly codeable 10 of the 17 times it was used across 4 out of 5 interviews. Synonym belief words proved more difficult. We provide one clear nearby belief word example in our table, above, and one example in our analysis, below, but focus primarily on learning orientation coding for ease of characterization.

A. Coding for learning orientation
We found that students sometimes described performance orientations aligned with fixed mindset when describing challenges. One such example is Kasee (pseudonym) who described a challenging context of attending a physics camp in high school hosted by experimental physics researchers at a local university. There is no interviewer question context to describe because Kasee volunteered this story before the interviewer asked their first formal question. Kasee describes the challenge of the physics camp in the following way:"...this was the summer after my sophomore year of school, so four years ago I guess, and it was all just way over my head.

I had no background and I really didn't know what was happening."
She went onto say that, "I think that was like too much, so I just got really afraid." Ultimately, she described abandoning her plans to pursue astrophysics as a result: "I was like, 'I could never do this.' Because I used to want to do astrophysics, and then I did [the summer camp] and I was like, 'Nope, this is not for me.'" According to Dweck (2000) a student displaying a performance orientation for learning will assess their ability in a challenging situation and give up if it appears their ability is too low. This directly aligns with Kasee's description of her challenging physics camp.
In our interview data we were also able to code for instances where a student displayed a mastery orientation to learning (growth mindset). For example, when the interviewer asks Julie "Have you had any experiences [in physics class] of…not doing very well in some case or trying to understand what happened or why?" she describes taking challenging exams in her current introductory physics course (see Table 1). In Julie's story we see a clear example of a mastery orientation because she is responding to a challenge by changing strategy rather than giving up.

B. Preliminary evidence for context-dependence
We also have some promising examples of how particular interviewer questions might prime students. Family story-telling questions often elicited more fixed mindset responses (see Julie, Fig. 1 and Kasee, below). Questions about improvement and being proud in physics classes often elicited more growth mindset responses. Our clearest example of context-dependence can be seen in excerpts from Kasee's interview. Bolded words correspond to evidence for categorization.

Kasee Context One: Feeling Proud in College Physics and Growth Mindset
When the interviewer asked Kasee what she was most proud of over her two semesters of college physics, she described: "I'm proud that I actually understood what was happening. Because even with no background…I'm really proud, like I worked really hard last semester to understand what was going on…So that's the most

important thing for me, is making sure I feel like I understand the material"
Kasee's dialogue can be categorized as displaying a mastery orientation because she notes hard work in the face of challenge (her physics course) and privileging understanding as more important than getting the right answer or a good grade.

Kasee Context Two: Family Roles and Fixed Mindset
During the same interview, within the interviewer question context about whether her parents were surprised by her chosen path of study she answered: "I don't think so, because I've always been very inquisitive, stuff like asking questions always. My  The storytelling context Kasee describes is with respect to family relationships: her parents. Kasee notes that she asks "questions always." She appears to be referencing younger moments in her life at home as the storytelling context, however, the transcript does not outlaw an interpretation of her statements as referencing the context of school science courses as well. The nearby mindset belief word Kasee employs is "science stuff." We also argue that she uses it in a fixed way since she uses "always" twice in describing what is connected to getting the "science stuff."

DISCUSSION AND CONCLUSIONS
While we have some preliminary evidence that supports Dweck's assertion that people are mixtures, we do not yet have consistent context-related patterns.
In addition, our examples of nearby belief statements that are "fixed" cannot be interpreted as problematic in the same way as strongly agreeing that "you have a certain amount of intelligence and you can't do much to change it" [9]. Further work will need to unpack students' implicit and explicit understanding of the relationship between intelligence, talent, synonyms, and nearby words. The latter two categories are important because talent and intelligence are used less frequently in colloquial student talk. We take up mindset as a construct to examine the strengths, weaknesses, and implications of categorizing students and their dialogue in this way. Students' educational contexts are clearly consequential, something we hope to unpack more through our context-dependent study.
Our main contribution to the literature with this short paper is to present the first analysis framework for examining mindset in interview data. Although our pilot study has limitations, our analysis suggests productive future places to follow up around context-dependence. Since peer and faculty messaging around mindset in STEM at large research universities often negatively affects students, particularly those already marginalized, [12,13,17], helping instructors change behavior to leverage productive context-dependent student beliefs will be key. Our pilot studies were also limited across gender (mainly women) and race (White); further work will necessitate examining more varied gendered and racialized experiences around intelligence beliefs [14]. Lastly, we highlight that issues of identity (e.g. "science person") can be seen in student dialogue. Future work will connect identity and mindset.