Process of transforming of an introductory mechanics lab course at Fort Lewis College

There are many recent calls to transform lab courses to better engage students in authentic scientific practices and develop their views on the nature of science. To address these calls and support local desire for an improved lab curriculum, we have begun to work with Fort Lewis College faculty, staff, and students to transform their introductory mechanics lab to improve student learning of experimental physics. Here, we describe our preliminary work in which we sought community engagement and input on goals and content for the labs. We did this through in-depth interviews with faculty, classroom observations, student reflections, and through the formation of a Student Advisory Council. From these sources, we developed a set of guidelines and consensus learning goals, which will inform the creation of the transformed lab course.


I. INTRODUCTION
Undergraduate laboratory classes provide students with the opportunity to engage in authentic scientific practices and develop their views on the nature of science. Recently, there have been many calls to transform lab courses from the physics community [1][2][3]. However, much of the assessment of the resulting impact on students' experiences in transformed labs has been conducted at large, research intensive institutions [2,4,5]. To better understand effective learning environments for all students at a broad range of institutions, we have begun working with Fort Lewis College to transform their introductory mechanics lab. Fort Lewis College (FLC) in Durango, CO is one of six designated Native Americanserving, non-tribal colleges by the U.S. Department of Education. It is one of the most diverse colleges in the nation-FLC has approximately 3,000 undergraduate students 30% of which are American Indian or Alaska Native identifying [6].
One of our most important goals, particular to the lab transformation at FLC, is to align with the mission and core values of Fort Lewis College [7] and develop labs that serve Native students. Although always good practice, it is particularly important for us to continually seek community assistance and input on goals and content for the labs given the agonizing history of research conducted on indigenous peoples and education practices tied to Western injustice and colonialism [8]. Many researchers and academics see their research projects having some "emancipatory goal" for an oppressed community and assume that they know best when working with other communities [8]. As non-native researchers, we acknowledge these wrong-doings and strive to work closely with Native students to place their experiences and needs at the forefront of this research. However, not wanting to put pressure on individual Native students to speak for all Native students at the college-FLC has over 160 tribal nations represented in their student body [6]-we seek supplemental support from FLC faculty who work closely with Native students and past research into challenges for Native students in higher education [9][10][11][12][13][14][15].
Here, we will discuss the methodology and results of our preliminary work, which includes in-depth interviews with faculty, classroom observations, collection of student reflections, and formation of a Student Advisory Council.

II. TRANSFORMATION PROCESS AND OUTCOMES
As part of the larger STROBE collaboration [16], we were asked by the faculty at FLC to assist in the transformation of the lab component of an introductory physics course focusing on mechanics. As we are coming from a different institution and institutional context, we began the project by defining two core principles to guide the process: (1) The transformation processes will be driven by data collected at FLC to develop a culturally responsive curriculum informed by our past research of best practices in labs and we will (2) continually engage all stakeholders throughout the transformation process, including students, staff, physics faculty and FLC faculty familiar with working with Native students outside of physics. To adhere to these principles, we started by spending a week at FLC in the Fall of 2019 to meet with faculty, staff, and students, as well as see the facilities and observe students in the lab. Based on input gathered from this visit and subsequent interactions during that semester, we developed a plan for the transformation. This plan included collection of baseline data through student reflections, interviews, and the E-CLASS survey [17,18] during Spring 2020. Additionally, we would create one new one-week lab activity and test it in the classroom during Spring 2020. Then, based on analysis of data collected and input from the test run of a new lab activity, we would design and implement the full series of 10 labs in Fall 2020-we note that due to the pandemic, this timeline has now been delayed.
Current Course Context. This course is the first calculusbased physics course in the department and is usually taken by sophomore physics and engineering majors, as well as some biology, chemistry, and geology students. Students taking the course are required to enroll in both the lecture and a lab section. The course has typically 40 -60 students enrolled with at most 20 students per lab section. The labs are taught by a faculty member and often assisted by undergraduate teaching assistants. The course has 10 labs that are three hours and fifteen minutes long. The first four labs are tutorials focusing on learning lab skills such as graphing and fitting exercises, uncertainty analysis, vector addition, and using measurement tools. The fifth lab is a practicum exam assessing these skills. The next four labs are more traditional covering friction, a simple pendulum, non-conservation of mechanical energy, and rotational physics. Students work together in groups of two or three during these labs and write group lab reports for each. The final lab is an exam that tests students on experimental skills using software and measuring tools.
Throughout the rest of this section, we will discuss the initial data collection-including faculty interviews and input from the Student Advisory Council-and outcomes from the analysis of these data.
Faculty Interviews. During our first visit to FLC, we met with five of the faculty who have taught, or may teach, this course. Our goals for the faculty interviews were to determine consensus learning goals and understand the challenges they faced when teaching this course.
During the interviews, we found that the faculty agreed on five main ideas: (1) The primary learning objective is for students to gain experience in the main practices of experimental physics, especially in data analysis. This includes being able to collect, analyze, interpret, and present data. (2) The lab component of the class should cover mechanics physics concepts such as forces, kinematics, energy conservation, friction and rotation. The lab topics do not necessarily need to be in sequence with the lecture material. (3) Students should enjoy the class. (4) The labs should help students gain confidence in doing experimental physics while (5) connecting what they have learned in class to real-world applications.
In addition to describing the broad goals for the new course, faculty also discussed some common challenges faced by the students and faculty at FLC. First, we heard that, although there is a large range of student preparedness, students often have less conceptual physics knowledge than is required to complete the labs. This has resulted in more prescriptive labs and lectures during lab time to get the students prepared for that day's activity. Another concern was how to assign a grade for the lab. We heard from the faculty that many of their students struggle with writing lab reports. This made grading written assignments challenging, as they were not sure how much weight they should put on grammar and formatting versus content. Lastly, faculty noted that students struggle in subsequent courses on material, in particular error propagation, that was covered in mechanics lab. This could be for a multitude of reasons, but one reason, that was suggested by both faculty and the student advisory council, might be that the formulas for error propagation are forgotten if they are not connected with a student's prior knowledge, namely, mathematics. We heard that measurement uncertainty was important, but formal error propagation with partial derivatives could be saved for a future class when students have stronger math experience.
Student Advisory Council. To engage students, who are critical stakeholders in this course transformation, we formed a Student Advisory Council (SAC). Our goal for forming this council of student advisors was to seek advice and insight as we develop the new mechanics labs such that they best meet the needs of FLC students. The council consists of six students, two of whom are Native identifying, who had taken the mechanics lab. Building and maintaining relationships with students is necessary to ensure the student voice is heard and remains an important factor when developing the labs.
The six students were identified and recommended by the faculty. The students were asked to dedicate no more than five hours per semester to meeting via video chat to discuss personal learning goals, as well as provide personal thoughts, feelings, and initial impressions about new labs. We provided monetary compensation to make sure to value their time.
Throughout the semester, we met with the students individually and as a group. From the students, we heard many things they enjoyed about the current course, but also a few frustrations. First, students enjoyed working closely with their lab partners, but struggled writing joint lab reports. They expressed that writing assignments had unclear expectations and it was difficult to balance the workload between them and their lab partner. From the faculty interviews, we found that faculty workload was a challenge-the course enrolls 40-60 students each semester and at a small liberal arts college, the faculty have to grade all of the assignments themselves. This had resulted in assigning group lab reports.
Additionally, students in the SAC reported some frustration that the labs did not help them on homework assignments or during the exams for the lecture portion of the course. Furthermore, none of the students mentioned experimental skills they learned, such as using Excel to compute means and stan-dard deviations of data. Together, this could stem from a mismatch of expectations of the students and the design of the course. Students appeared to view the purpose of the lab was to reinforce conceptual physics understanding, which could help them on homework or exams, rather than to learn experimental skills, such as data analysis.
Some of the students expressed the desire for more agency. Many of the labs had prescriptive step-by-step instructions for both collecting and analyzing the data. Conversely, some of the students expressed that they were not prepared for all of the labs and appreciated the prescriptive nature of the lab manuals and the lab lectures. Finally, many students felt like they were not interested in the labs and wished they had more relevance to their lives.

A. Outcomes
Together, we used these desires and concerns from the faculty and students to develop measurable learning goals, as well as guiding principals that informed the development of a wind turbine lab to implement as a trial lab during the Spring 2020 semester.
Learning Goals and Guiding Principles. We took the five consensus goals we determined in the faculty interviews as well as insight from our student advisory council to develop measurable learning goals. The outcome was nine goals, see Table I. The first three learning goals are focused on student affect and their views about the nature of experimental physics. Goals 4 -8 discuss various aspects of data analysis and student understanding of measurement uncertainty. Instead of students memorizing complex equations for error propagation, knowledge which they did not seem to carry into future classes, we hope to focus on developing student understanding that no one measurement is correct, all measurements have uncertainty, and repeated measurements form a distribution from which one can determine the mean and spread. The last goal describes expectations for student writing assignments-instead of writing full lab reports, the goal will be for students to gain a mastery of writing results and discussion sections. In addition to the learning goals, it was equally important to define guiding principles that were informed by the student advisory council. Some of the guiding principles included: (1) Students will engage in teamwork to solve problems, (2) we will not have verification labs where students measure a well-known quantity, (3) labs will be designed with Native cultural context, and (4) the amount of time outside of class the instructors put in to the course will be not be increased.
Recommendations to Achieve Goals. To help achieve some of these goals, be consistent with guiding principles, and address concerns of the students, we proposed having short writing assignments that focus on the results/discussion section rather than full lab reports. Not only would this address learning goals 4 and 9, it would reduce the load on both the students and faculty, while giving clearer expectations for the writing component of the class. We also proposed creat- E-CLASS affect items and student reflection responses 4 Students should be able to create a publication quality graph of their data.
Course Artifacts 5 Students should be able to collect repeated measurements and describe the distribution of measurements using means and standard deviations.
Course Artifacts 6 Students should be able to estimate the uncertainty of a single measurement based on the measurement system apparatus. Course Artifacts 7 Students should be able to use the equations for propagating uncertainties (addition/subtraction and multiplication/division) to determine the uncertainty in a calculated quantity from uncertainty in a measured quantity.
Course Artifacts 8 Students should be able to compare the results of their measurements to other measurements and determine the level of agreement based on the uncertainty.
Course Artifacts 9 Students should be able to produce a results/discussion section that synthesizes key findings from collected data, presents those findings using appropriate graphing techniques, and reflects on the relationship between those observations and their understanding of physics.
Course Artifacts ing pre-lab videos with embedded questions to help prepare students before they arrive in lab [19], while eliminating lab lectures to give students more time with the lab activity. Additionally, we suggested that the labs will consist of mechanics concepts, but will not be directly linked to the lecture in order to avoid student confusion about the learning goals of the lab, which are experimental skills. The lab goals of developing experimental skills will now be framed explicitly from the first day of the class. Development of a Wind Turbine Lab. Based on these learning goals and guiding principles, we began developing a lab to test and observe during the Spring 2020 semester. Our goal of deploying a single lab during the semester was to observe how students engaged in various types of discussions and how they responded to making their own decisions during the lab. We decided to make a lab that would be the first lab in the 10 lab series during the final version of the course and focused on learning goals 1-3, 5, and 8 (Table I) [20]. In order for students to see and describe the distribution that is formed from repeated measurements and describe that distribution using means and standard deviations, we chose to use a wind turbine with electronic acquisition of the voltage output of the generator. Furthermore, wind turbines have many interesting mechanics elements which could be explored in later labs and they are relevant to extremely rural locations in the US such as Navajo Nation that rely on off-the-grid renewable energy sources.
Throughout the lab there were "Record" sections, where students were asked to record data or answer questions in their lab notebooks, followed by "Discussion" sections, where students were asked to discuss questions either with their lab partner, with one other group, or with the entire class. We collected the lab notebooks and observed the class to see how students responded to these various scenarios and questions.

III. DATA COLLECTION AND OUTCOMES
We conducted classroom observations during the new the wind turbine lab. Our goals were threefold: (1) to test out the lab itself and see the technical challenges of implementing a new lab, (2) to observe how students responded to making procedural and analysis decisions during lab, and (3) to observe students engagement with their lab notebook and discussions with their lab partner, other groups, and the class as a whole. Here, we will refrain from discussing (1) but focus on (2) and (3) and how they impacted our decisions for the transformed course.
In addition to the classroom observations for the one lab, we asked students reflections questions after all ten labs during the semester via an online survey. The goals of the reflection questions were to both inform the transformation process and provide baseline data to measure the impact of the transformation.
Decision Making in Wind Turbine Lab. Although the lab procedure was fairly prescriptive in nature-providing students with step-by-step instructions for collecting the data, the data analysis was open-ended. After taking repeated measurements, students had to decide how to report the voltage (e.g., mean versus median, standard deviation or standard error of the mean). The first time students reported the voltage, they were not prompted to report the error. Even having three prior labs that focused heavily on measurement uncertainty, almost all the students reported the mean voltage, but no groups reported the uncertainty in their measurement. This was contradictory to what many students reflected after the lab two weeks prior where they conducted an Uncertainty Analysis lab. In the reflections after the Uncertainty Analysis lab, students were asked, "Did today's lab change your views on measurement uncertainty?" The 19 student responses were coded by AW and HJL. (From these codes, we calculated the Cohen's kappa statistic [21], to be 0.81, which is considered almost perfect agreement [21].) We found that the majority of students (14 out of 19) said that the lab had changed their views on measurement uncertainty. Three students said that they learned all measurements have an uncertainty, four students learned to calculate uncertainty and five students reflected that they learned why error analysis is important to consider. Yet, reiterating the sentiments of the faculty during their interviews, the students did not implement the knowledge gained from the Uncertainty Analysis lab during the wind turbine lab two weeks later.
One hypothesis for this discrepancy is that the Uncertainty Analysis lab seemed to introduce/reinforce the idea that measurement uncertainty is based on human error or limitations from the measurement equipment. In the reflections, students were asked to rank their agreement with the following statement on a 5-point Likert scale, "Every measurement has an associated uncertainty." And then asked to, "Explain your choice." The 19 student responses to the "explain" prompt were coded by AW and HJL, resulting in a Cohen's kappa of 0.83. Almost all the students (16 out of 19) agreed that all measurements have an uncertainty; however, only five of the students discussed random error as being the cause of this. Instead, eight students discussed the precision of the measuring tool being a limiting factor and eight students reflected on human error being a cause for all measurements having an associated uncertainty. The measurement error during the wind turbine experiment was dominated by random error due to turbulence in the air. Since students were less familiar with this type of error, we propose that might have been why they did not think to report it during the lab.
Discussions During Wind Turbine Lab. Prior to the Wind Turbine lab, we had heard from FLC faculty and students that class discussions can be challenging, particularly for Native students. Even though FLC is a Native-serving institution, it still remains predominantly white [6] and given the history of oppression toward Native peoples in the United States and at FLC, as well as discourse differences, and issues of trust [9], it is not surprising that some Native students would be more reserved during classroom discussions. As a result of this, we designed the trial Wind Turbine lab to have various levels of discussions-individual recordings in the lab notebooks, discussions with lab partners, discussions with other groups, and class discussions, to better observe how students engaged in these scenarios.
We observed that even though it was their first time using lab notebooks, students very actively engaged with their notebooks. Likewise, most students easily discussed the questions with their lab partner and worked together. However, the discussions with other groups and class discussions were quite different. The lab class has five benches, each bench has four seats and two setups. The discussions with other groups were easy for students who worked at the same pace as the lab group at their table, but when two groups at the same table worked at different paces, the partners who finished first were very resistant to leaving their table to find a group that was finished even when prompted by the instructor. Whole class discussions were mostly silent. Students waited for the instructors to lecture and some students actively hid behind their wind turbines to avoid being called upon. We realized to have productive team work and discussions we would need to build students comfort and confidence from the first lab.
Guidelines for Class Discussions and Active Engagement. Research into belonging shows that students from underrepresented backgrounds may feel isolated within the college environment leading them to feel less engaged in college courses [10,[22][23][24]. However, active learning has been shown to significantly increase learning over traditional lectures [25,26] since dialogue is a means for students to critically analyze, evaluate long-held assumptions, and help each other learn [22,27]. Active learning is a core part of the lab design that we want to highlight, but in order to do this in an equitable way, we will need to employ some strategies to foster an inclusive environment. For example, students will always be prompted to write their ideas in their lab notebook prior to engaging in whole class discussions and class discussions will be highlighted in the lab manual so they are not a surprise to students.

IV. CONCLUSIONS AND FUTURE STEPS
We have outlined the preliminary steps that were taken to inform the transformation of the introductory mechanics lab at Fort Lewis College. We first engaged the stakeholders by conducting faculty interviews and forming a Student Advisory Council. We used responses from faculty and students to develop consensus learning goals, guiding principles, and a trial lab, which we implemented in the Spring 2020 term. Additionally, we collected student reflections after each of the labs and observed the trial Wind Turbine lab. We used all of the data collectively to determine challenges currently faced by students and faculty and determined possible solutions to them.
Finally, we would like to acknowledge that due to the COVID-19 pandemic, two of the labs during the Spring 2020 term were moved to remote instruction and very few students responded to the reflection questions during that time. Furthermore, we were also unable to conduct student interviews at the end of the semester. We reflect back to our first core principle-have a process driven by data from FLC. To develop these labs for and with the students, we have decided to delay the transformation until we can safely engage in person.