Using observations of Universal Design for Learning to enhance post-secondary STEM teaching practices

Universal Design for Learning (UDL) is a framework for instruction enabling instructors to impact learners with varying cognitive strengths and skills by emphasizing the need for multiple means of representation, expression and engagement. UDL is increasingly used in many post-secondary programs; however, this framework is relatively unknown to STEM instructors. Research focusing on UDL in post-secondary has primarily centered on teacher and student perspectives. This project uses observations to describe the extent to which student-centered active learning STEM courses implemented UDL practices. Observations were conducted during introductory physics SCALE-UP and chemistry inquiry laboratory courses using a protocol based on the UDL framework. Observers coded the prevalence of 31 practices in four categories: introducing and framing new material; content representation and delivery; expression of understanding; and activity and student engagement. These observations allowed researchers to identify specific areas of needed improvement and will be used to facilitate STEM instructors incorporation of UDL in future lessons.


I. INTRODUCTION
Recent calls highlight the need to recruit additional STEM (science, technology, engineering, math) majors to support the economic growth of the country [1].Students with disabilities (SWD) represent an underserved pool of potential STEM majors, with many SWD dropping out of the STEM major before completing their degree [2].SWD face many barriers at large universities, including large class sizes, the fast pace of instruction, lack of scaffolding in the curriculum, precision of the content, and the pedagogical approach [3].While traditional institutional reform focuses on system-level changes in instructor behavior and support, such reforms have not had an immediate wide-scale impact on the attrition rate of STEM majors with disabilities [4].
Universal Design was first conceptualized in architecture as "the design of products and built environments that are usable by all people, to the greatest extent possible, without the need for adaptation of specialized design" in 1988 [5].In the late 1990s, Universal Design was applied to education with the creation of frameworks such as Universal Design for Learning (UDL), but was mostly applied at the K-12 level.In 2008, the U.S. Department of Education passed the Higher Education Opportunity Act (Public Law 110-315), which was the first official government document to define UDL and its role in post-secondary education [6].However, nearly ten years later, UDL has still been mostly applied in K-12 and post-secondary courses outside of science [5], and research in postsecondary STEM has been limited.
UDL focuses on three key principles: multiple means of representation, multiple means of expression and action, and multiple means of engagement [4,7,8].Instead of trying to support students' learning by using accommodations or making modifications to fit the teaching, UDL provides a template to change the actual learning environment to fit students' needs [9].While design with SWD in mind, UDL can create multiple ways for all students to access the curriculum [10].
In this study, we used observations of physics professors and chemistry teaching assistants to learn how common research-based, student-centered STEM pedagogies related to UDL and to develop recommendations for how instructors can use UDL to better meet the needs of all students.

II. METHODS
We conducted observations of two physics professors and two chemistry teaching assistants using the Universal Design for Learning Instructional Observation Instrument (UDL-I0I) [11].Observations occurred in physics SCALE-UP classrooms and chemistry inquiry-based laboratory courses.Each observation lasted three hours.Categories for scoring included: introducing and framing new material; content representation and delivery; expression and understanding; and activity and student engagement.The UDL-IOI was designed to pinpoint areas in need of improvement for instructors based on the UDL principles.Observers score courses on a four point Likert scale: 0 = no evidence of UDL; 1 = incomplete evidence of UDL in environment; 2 = UDL is occurring; and 3 = dynamic, interactive UDL.
Two researchers, one from the physics department and one from the exceptional education department, conducted the observations.Observations were initially done in tandem to ensure fidelity and inter-observer agreement.Agreement was reached when the researchers' scores were consistently within plus or minus one point of each other.After agreement was reached, the researchers completed at least three observations of three instructors and two observations of the fourth.Data was then analyzed and discussed with each professor or teaching assistant in the context of a one-week summer workshop in order for the instructors to plan changes to align their teaching methods more closely with the UDL framework.

III. OBSERVATION DATA
We present the observation results for each category of the UDL-IOI and discuss common areas of strengths or weaknesses across the observations.Each category consisted of several sub-categories.We present the average score per instructor on each sub-category and the standard error across observations of each instructor supplemented with examples from our field notes.

A. Introducing and framing new material
When introducing new material, it is important for an instructor to understand what their students already know.UDL principles focus on creating a level playing field for all students, no matter their exceptionality.When introducing and framing new material in a lesson, it is imperative for instructors to highlight key information and build on student background knowledge [12].
Six sub-categories focus on how instructors introduced and framed new material: a) assesses background knowledge prior to introducing new material; b) supports linking background knowledge to new knowledge; c) highlights what is important for students to learn; d) supports understanding of big ideas and critical concepts; e) uses questions supporting understanding or inquiry; and f) identifies potential misunderstandings/alternative conceptions.
This category was the greatest strength for the instructors in this study.As shown in Figure 1, all instructors scored higher than one, with the average being a score of two (UDL is occurring).For example, during the observations we saw instructors review answers to prior assessments to check for student understanding and use this information to discuss common areas of difficulty while students were performing experiments.
The sub-categories showing the most room for improvement were assessing background knowledge prior to introducing new material and highlighting what was important for students to learn.For example, we observed some instructors make use of an "essential question" at the beginning of the lesson, but it was rarely referenced again later in the lesson.It is important to note that the observers did not have access to the used in the chemistry lab; the teaching assistants may have used these assignments to assess students' background knowledge.In the future, observers will access these pre-lab assignments to strengthen claims based on our observations.

B. Content representation and delivery
The guidelines for content representation and delivery focus on preparing a lesson for any type of student who man enter the classroom.For example, instruction aligned with the UDL framework providess students who are not native English speakers opportunities to access the curriculum and provides support for students who may struggle with vocabulary.This principle encourages instructors to scaffold the information presented to students to effectively engage them in the lessons.
Nine sub-categories focus on content representation and delivery: a) supports multiple levels of content understanding (e.g.novice, intermediate, expert); b) presentation of information allows for customization; c) instruction allows for visual display of information; d) instruction allows for alternatives for auditory information; e) supports options for multiple languages; f) supports understandings of relationships across disciplines, settings, or concepts; g) clarifies content-specific vocabulary, symbols, and jargon; h) clarifies content-based syntax and structure; and i) highlights options for self-directed clarification of vocabulary and symbols.

FIG 2 Distribution of participant scores on content representation and delivery
This category showed the most need for improvement for our instructors, with an average score of one (incomplete in environment) across sub-categories and many scores of zero (no evidence of UDL) on sub-categories a, c, d, e and i (see Figure 2).For example, instructors in this study did not actively provide alternatives for students who spoke different languages, had a hearing impairment, or had a vision impairment.During the summer workshop, the research teams demonstrated several internet-based options to assist English Language Learners (ELL) or students that have hearing or vision impairments the instructors could incorporate in their courses.

C. Expression of understanding
Expression of understanding provides different outlets for students to showcase their work and show comprehension of content taught in class.Conventional classrooms typically focus heavily on paper and pencil tests and quizzes as means for students to demonstrate understanding.UDL encourages instructors to see past traditional methods and allow students opportunities to express their understanding through methods that meet students' strengths, such as oral presentations, poster presentations and written essays, or more creative options, such as creating a skit or putting information learned in class to a song.
Seven sub-categories focus on expression of understanding: a) allows options for learners to express understanding in a variety of ways; b) provides access to a variety of tools and/or technologies that allow students to express their understanding; c) builds competencies in use of multiple options for expressing their understanding; d) provides options that guide students to plan, develop strategies, and/or goal-setting that promotes expression of understanding; e) the environment facilitates management of information and resources to achieve desired learning outcomes; f) intentionally provides supports for students' problem-solving and critical thinking abilities; and g) facilitates student self-monitoring of progress.
The instructors in the study had an average score of one (incomplete evidence of UDL in environment) on most subcategories in expression of understanding.Overall, instructors showed the most room for improvement on building competencies in use of multiple options for students to express their understanding (see Figure 3).During observations, the main source of showing understanding was paper/pencil tests and quizzes with no formative assessments throughout the lesson.Allowing students multiple opportunities to choose how they express their understanding provides them with the skills necessary to pick options that best display their comprehension and abilities.

D. Activity and student engagement
While all principles of UDL are important, if students are not engaged in the lesson, they will not learn the content presented.Engagement is critical to students' processing skills, taking key information, and putting it into long-term memory.The longer students are engaged in a lesson, the longer they are able to sustain effort and focus on the information provided during the class [13].In the UDL-IOI, the activity and student engagement category also focuses on progress monitoring and how an instructor ends the lesson.Student self-monitoring is a vital part of self-growth and determination.By allowing students to monitor their own progress, they are taking ownership of their work.Instructors also need to provide formative progress monitoring to ensure students are comprehending the content and are ready to progress to the next topic.Closure at the end of a lesson reiterates the important ideas of the lesson and focuses student attention again on the key details learned during the class time [14].
Nine subcategories focus on activity and student engagement: a) promotes learner choice and selfdetermination while engaging with the content; b) provides a variety of activities relevant to all learners; c) promotes FIG. 4 Distribution of participant scores on activity and student engagement sustained effort and focus; d) encourages learners' use of strategic planning to complete instructional tasks; e) encourages collaboration and communication among learners; f) supports multiple levels of challenge; g) provides for self-reflection and self-assessment; h) provides formative progress monitoring and content checks; and i) provides closure that reiterates big ideas and instructional purposes.
The average participant score was one (incomplete evidence of UDL in environment) across each of the nine sub-categories with strengths shown in sub-categories related to collaboration and communication and formative progress monitoring.During observations, instructors used "clicker questions" where students would answer questions from the PowerPoint presentation and their answers would be shown on the board once everyone had answered.This provided opportunities for discussion amongst the students when there were disagreements in the answers.The two categories that show the most room for improvement related to supporting multiple levels of challenge and providing for self-reflection and self-assessment (see Figure 4).Instructors can provide multiple levels of challenge by creating lessons that scaffold content based on learner level.Students should also be able to self-reflect and self-assess throughout a lesson.This can be in the form of rubrics for assignments with clear outlines, self-evaluation rubrics or other reflection that allows students to monitor their progress toward personal goals.

IV. PARTICIPANT GOALS
Instructors received their average scores based on the observation tool with descriptions and examples for each subcategory.They then learned about UDL, SWD, and worked on lessons for the Fall 2017 semester integrating the UDL framework based on their previous scores.Instructors were not asked to incorporate all the principles at once throughout the whole semester, but rather to choose general principles to use in lessons and to choose a specific week to work on significant alignment.Instructors submitted online self-reflections at the end of each workshop day describing what they learned and how they would use what they learned in creating their lessons.
Participant 1 reflected on the way students absorb information.His goal is to provide multiple avenues to students so they all can learn via their preferred means.Participant 2 is looking to lay out expectations more clearly throughout his lessons and to make his lesson more accessible to all students by using various forms of technology.Participant 3 would like to foster better communication during the lecture portion of his lessons to increase student engagement.He would like to devise strategies to maintain student focus throughout the class period.Participant 4 will be focusing her efforts on encouraging class participation and discussion for the unmotivated students in her class.She would also like to focus on her delivery of the content, namely making it accessible to all students.

V. DISCUSSION
Students with disabilities often drop out of STEM majors due to confounding variables such as instructor approach to content [3].The Universal Design for Learning framework can alleviate some of the barriers students face in STEM majors.Observations with the UDL-IOI can help instructors pinpoint specific areas where UDL-based principles can improve access to the curriculum for diverse learners.
Even though the instructors in this study were using research-based, student-centered pedagogies, there was still much room for improvement.The main areas for improvement were in the category of content representation and delivery.Providing options for student to engage in lessons will allow learners to work to their strengths and engage deeper in the content and curriculum.We will continue to use the UDL-IOI to observe the instructors as they implement their planned changes to assess growth from previous observations.

FIG 1 .
FIG 1. Distribution of participant scores on introducing and framing new material