Physics instructors’ views about supporting learner variation: Modifying the Inclusive Teaching Strategies Inventory

Every person has abilities across a multidimensional spectrum; however, previous research has indicated that postsecondary faculty are unaware of how to support students with a broad range of abilities in their courses and receive little training about inclusive teaching strategies. On average, STEM faculty have demonstrated more negative views toward students with disabilities than instructors from other disciplines. As such, we want to better understand physics instructors’ beliefs about people with disabilities and their inclusive teaching practices. The Inclusive Teaching Strategies Inventory (ITSI) was developed to measure postsecondary instructors’ beliefs and practices related to disability and supporting people with disabilities across disciplines. Through a pilot administration of this survey, we found that STEM faculty experienced difficulties in responding to the survey. Thus, we modified the ITSI for use with STEM faculty. We present our modification process, describe specific modifications made to the ITSI, and discuss preliminary interview and survey data.


I. INTRODUCTION
All students vary in terms of their needs, abilities, and interests [1]. However, studies have shown that postsecondary faculty lack awareness of the legal requirements related to accommodations [2][3][4][5][6], lack knowledge of inclusive pedagogies [7], state that they do not feel prepared to teach students with disabilities [8][9], and want more training related to accessibility [10][11][12][13]. Specifically, STEM faculty have been shown to be less amenable to the use of accommodations in their courses and to hold more negative beliefs about students with disabilities than their counterparts in other academic disciplines [14][15][16][17]. Also, popular research-based introductory physics and chemistry curricula have been shown to not be designed to support learner variation [18][19]. Since students with diagnosed disabilities represent approximately 20% of undergraduate students in 2019 [20], the lack of faculty preparation and curricular support imply that there are students enrolled in physics courses who are not being supported or centered in the curricula. This can lead to students with disabilities feeling like they do not belong in STEM [21] which has been shown to affect retention in their degree programs [22]. The purpose of this paper is to present: an instrument, called the Inclusive Teaching Strategies Inventory, that can be used to measure faculty perspectives about and use of inclusive teaching strategies; modifications we made to the survey to tailor it for the STEM community; and preliminary data collected using the survey. Accurately measuring faculty member's beliefs and attitudes about people with disabilities can be the first step toward making STEM more accessible and supportive of learner variation.

II. INCLUSIVE TEACHING STRATEGIES INVENTORY (ITSI)
The Inclusive Teaching Strategies Inventory (ITSI) is a self-report survey that measures faculty attitudes and beliefs about people with disabilities, accommodations, and inclusive teaching strategies [23]. It is composed of 32 prompts that describe instructional practices; for example, "allow students to digitally record (audio or visual) class sessions." Participants respond to two questions for each prompt: 1) "I believe it's important to…" with response choices ranging from strongly disagree to strongly agree on a 6-point scale; and 2) "I do this…" with response choices on a 4-point scale (never, sometimes, usually, always) [24]. The first question probes instructors' beliefs, and the second question probes the instructors' actions in their courses. The survey also includes questions about the participants' demographics (i.e., race/ethnicity, gender, professional role) and disability-specific questions (i.e., personal experience with disability, professional experience with people with disabilities, disability-specific training).
The ITSI is composed of seven factors: 1) Accommodations, 2) Accessible Course Materials, 3) Course Modifications, 4) Inclusive Lecture Strategies, 5) Inclusive Classroom, 6) Inclusive Assessment, and 7) Disability Laws and Concepts [24]. The survey was developed to be used by all faculty at the postsecondary level regardless of their disciplines. As such the prompts are general and do not include discipline-specific information or practices.

A. Validity and reliability evidence
Multiple studies have investigated the validity and reliability of the ITSI. The developers provide evidence for construct validity by building the items based on theoretical frameworks. They also provide evidence of content validity through interview data collected with experts. The developers took items from previous surveys, which provides evidence for the content validity. Finally, the developers provide evidence of discriminant and convergent validity through exploratory factor analysis [23]. This initial evidence was collected with data from "faculty at a mediumsized public research university located in the Pacific Northwest" (p. 45) [23]. The validity of the survey was subsequently investigated via a cross validation exploratory and confirmatory factor study with faculty from the United States, Spain, and Canada. Findings provided evidence to support the hypothesized seven factor structure [26]. The evidence for the reliability of the survey was primarily generated via Cronbach's alpha for the factors of the survey [23][24][25]. Most recently, Lombardi, Murray, and Dallas analyzed ITSI responses from faculty at two universities in the United States and found Cronbach's alpha values for all seven factors that ranged from 0.7 to 0.87, which provides evidence for the reliability of the survey prompts [25].

A. Impetus for modifications
As a pilot, we asked physics graduate students and physics and chemistry faculty members to take the ITSI and share their thoughts about it. Afterwards most expressed that they did not know how to respond to some of the prompts because some of the prompts specified a student population (e.g., provide copies of my lecture notes or outlines to students with documented disabilities) while others did not. The pilot participants wanted to indicate what population of students they had in mind for each prompt because their responses would vary depending on the population. For example, they may think a practice is important for students with disabilities but not important for all students. Also, because the ITSI was designed to be general and not disciplinary-specific, some of the prompts did not readily translate to the STEM classroom. For example, in a typical introductory chemistry or physics course, students are more frequently asked to solve problems outside of class than to participate in a discussion board; thus, questions pertaining to discussion board participation would not be frequently relevant in STEM courses. For these reasons, we decided to modify the ITSI in order to be applicable in a STEM context.

B. Modification process
Our modification process took part in multiple phases. Specifically, modifications were made after: the pilot distribution, interviews with physicists, and discussion with the developer. Expanded examples to include other learning management systems. 2 15. Repeat a student's question back to the class before the instructor answers the question.
Changed wording to emphasize that instructor repeats the question. 2 16. Begin each class session with an outline/ agenda of the topics that will be covered. Added 'online forum' example. 2 21. Cover course information in multiple formats (e.g. lecture, text, graphics, audio, video, hands-on exercises).
-23. Use a variety of instructional formats in addition to lecture, such as small groups, peer assisted learning, and hands on activities.
- 25. Allow students to demonstrate the knowledge and skills in ways other than traditional tests and exams (e.g. written essays, portfolios, journals). -Superscripts (table and text) refer to when changes were made: 1 after the pilot distribution, 2 after the interviews, and 3 after discussion with a developer. * indicates student group-specific information was removed. † indicates third question was added. Table I contains the prompt-specific changes made to the ITSI. We also made three non-prompt-specific changes. The first and most substantial change we made was to add a third question asking participants to indicate the student group they had in mind while responding to the first 13 prompts. Specifically, the question is "I would do this for…" with four response choices (no students, only students with disabilities, students who need it, and all students; indicated with a dagger in Table I) 1 . Secondly, we removed questions that were now redundant due to the addition of "I would do this for…" (indicated with an asterisk in Table I) 1 . Because our other changes had lengthened the survey, we removed questions that were about disability laws (e.g., "I am confident in my understanding of the legal definition of disability" with a response scale from strongly agree to strongly disagree) and on-campus disability services (e.g., "I know a Disability Services office exists on this campus") to more specifically focus on inclusive teaching strategies. Finally, we added an attention check question (i.e., "Attention check (mark: strongly agree and never)") 2 .

IV. PRELIMINARY FINDINGS
After we modified the ITSI, we implemented the modified version in a pilot study. We collected survey responses at two American Physical Society (APS) section meetings; all meeting attendees were eligible to participate, including undergraduate and graduate students, postdoctoral scholars, faculty members, and industry professionals. Participants received a gift card as compensation for completing the survey. A total of 19 complete responses to the survey were collected. However, six of these participants did not accurately complete the attention check question and therefore were not included in the analysis presented here.
Eight of the participants identified as male, four as female, and one as nonbinary. Four participants were students, eight were college or university faculty, and one was an industry member. We asked participants about their personal experiences with disability and allowed them to report multiple experiences. Ten participants stated they have worked with or taught students with disabilities, eight said they have a personal contact (e.g., family member or friend) who has a disability, one stated they have a disability, and two stated they have no personal experience with people with disabilities. Below we present preliminary findings for the "I would do this for…" question because the addition of this question was the most substantial change we made to the survey.

A. "I would do this for" responses
Participants responded to three questions (focused on beliefs, actions, and population) for each of the first 13 prompts. The participants' responses to the population question are shown in Table II. Overall, few participants chose the "no students" response choice for the first 10 prompts; however, this trend changes for the last three prompts. The last two prompts probe the difference between accommodations and modifications [27]. Accommodations are required by federal law [28][29] and include changing the learning environment to increase access while keeping the complexity and rigor at the same level. On the other hand, modifications are changes that lower the complexity and/or rigor [30]. The view more aligned with disability experts is that accommodations should be made to promote access but modifications should not be made in order to keep the level of rigor the same for people with and without disabilities. Thus, it makes sense that more participants chose the "no students" response choice for the last two prompts, indicating a more expert-like understanding.
The prompt "allow students to complete extra credit assignments" (11) also had more participants that selected the "no students" response choice and had a bimodal distribution of responses (i.e., nearly half of the participants chose "no students" and most of the other half chose "all students"). This is interesting because Lombardi et al. consider extra credit to be a modification and thus, not appropriate in all circumstances [25]. However, they suggest measuring faculty's views about extra credit provides insight about areas where faculty may be more or less willing to be flexible about course requirements.

B. Participants' views of "only students with disabilities"
As a follow-up to the first 13 prompts, we asked participants who they included in the "only students with disabilities" response category. Participants were given four non-mutually exclusive response choices: I. Students registered with the disability services office on campus, II. Students not registered with disability services office but who have a diagnosed disability, III. Students who identify with a disability (i.e., undiagnosed), and IV. Other, please specify. Five participants selected only option I, two selected only option III, four selected I and II, and two selected I, II, and III. Not surprisingly, most participants (11/13) included students registered with the disability services office in this category; the two participants who did not select option I selected only option III (students who identify with a disability (i.e., undiagnosed)) and may have assumed this category included students with disabilities. We plan to tweak this response option for future surveys.
Even with our small pilot sample size, there is variation in responses between participants, which implies that participants are not including the same subset of students in the "only students with disabilities" response category. This has implications for the validity and reliability of the survey because participants are interpreting the response choices differently. One way to mitigate this threat to validity and reliability in the future is to provide an operationalized definition of "only students with disabilities" at the start of the survey to orient the participants so that they can have a common interpretation of the response choice. Another option is to continue collecting participant responses to this question ("Who is included in the "only students with disabilities" response choice?) and only compare responses from participants with common interpretations of this response choice.

C. Participants' views of "students who need it"
We also asked participants who they included in the "students who need it" response choice in an open-ended format. As such, there was more variety in participants' responses; however, there were a few common themes in their responses, which we present here.
Six of the participants stated that they included students who express a need to them or students who self-identify as needing an accommodation. For example, participants stated "people who would express that they want this thing" and "students who self-identify to instructor." These responses indicate that participants include students who communicate with the instructor about their additional and/or alternative needs in the "students who need it" category. This can be particularly problematic because not all students want to disclose their needs and/or disability due to stigmatization they experience [31]. Courses should be designed to support all learners inherently and not rely on students' disclosure.
Another common participant response (3/13 participants) was including students with extenuating circumstances outside of the classroom. For example, participants stated "students that had extenuating circumstances" and "need is very loosely defined and may not be a disability but based on life circumstances (family needs, work, athletics, student activity groups)." Another participant added a temporal aspect to their response; "students going through temporary times of hardship." An interesting response was that participants included students with a "valid" excuse in the "students who need it" category. For example, participants stated "students that had extenuating circumstances and could demonstrate it" and "students who make a request to me and provide a 'valid' reason. 'Valid' is subject to my interpretation, which is why I generally reserve the large accommodations for those registered with disability services." These responses can be problematic for the reason the second participant stated; it can be very difficult for a faculty member to identify and/or determine if an excuse is valid. Each person has their own set of life experiences (i.e., positionality) that color how they view the world. If an instructor does not have the same types of life experiences (such as disability, family, or financial) as the student requesting an accommodation (which is unlikely), then the instructor may find it difficult to determine what is and is not a "valid" excuse.
Finally, other participants stated that the category would include people whose learning would be significantly impacted by the accommodation. For example, participants stated "students who it seems like the practice would substantially impact their learning and/or course performance" and "for whom I feel an accommodation would be both fair and helpful." In an ideal course, all instructors could support all students but instructors are limited in time and resources. This perceived lack of time and resources could be the reason why these participants choose to make accommodations that will give them the "biggest bang for their buck." The physics education community should work to try out and share practices that support instructors in accommodating the variety of life hurdles that may hinder students' ability to demonstrate their understanding of physics.

V. NEXT STEPS
Through this process we modified the ITSI to be more applicable to STEM courses. In the future, we plan to increase our sample to include more physics instructors to investigate their views about inclusive teaching strategies and to allow for an evaluation of the validity and reliability of the modified survey. Specifically, we are interested in investigating the relationship between instructors' selfreported importance of practices (i.e., response to the "I believe it's important to…" question) and implementation of the practices (i.e., response to the "I do this…" question). For example, it is interesting to see the rates at which instructors strongly agree that a practice is important but indicate that they never implement the practice in their class which could be related to limited instructor time and resources. Additionally, we are interested in analyzing the patterns of who instructors would implement the inclusive teaching strategies for in their classes. For example, whether there are practices that instructors only implement for students with disabilities (which would be accommodations instead of inclusive teaching strategies), while there are other practices that they implement for all students.

ACKNOWLEDGMENTS
This work is supported by NSF DUE 1612009 and NSF HRD 1750515.