Museum-based physics education research through research-practice partnerships (RPPs)

MOXI is an interactive science center focused on physics topics such as forces, energy, sound, light, and magnetism. MOXI’s exhibits and education program are informed by Physics Education Research (PER) and the Next Generation Science Standards (NGSS). As a result, MOXI is an outstanding laboratory for research on how people learn physics through interactive experiences and how best to support this learning. However, conducting research in public spaces with diverse audiences differs from classroom based research. These differences provide both opportunities and challenges. Effective research and program design requires multiple types of expertise including content, research design, and informal environments. In MOXI’s ﬁrst two years of operation, we have conducted research across a wide variety of participants and topics through a research-practice partnership (RPP) model. This paper focuses on establishing RPPs and methodological considerations when conducting research in informal science education settings such as interactive science centers.


I. INTRODUCTION
Interactive museums and science centers in which visitors explore physics phenomena are rich contexts for researching physics learning.However, conducting research in science centers and other "informal" (out-of-school) education spaces, differs considerably from classroom based research, creating both new opportunities and challenges.Such research requires multiple types of expertise including physics content, research design, and informal environments as well as resources including access to informal spaces, and access to academic journals and analysis software.Typically, neither informal educators nor physics education researchers have access to such diverse areas expertise and resources.Partnerships between researchers and museum practitioners leverage the expertise and resources of both informal educators and researchers to productively investigate such dynamic environments.The term partnership is used broadly to describe many ways that researchers and practitioners work together in productive activities.Researchers may conduct evaluations in education settings to inform best practices, conduct research in classrooms to understand student learning, or contribute their content expertise to projects or activities.Typically, these are temporary arrangements around a particular project and are mostly one-directional: one partner is helping the other accomplish a goal.
In this paper, we present a model of partnership referred to as Research-Practice Partnerships (RPPs) [1] [2] as a mechanism for sustainable research and development in complex learning environments.In contrast to other types of one-directional partnerships, Research Practice Partnerships (RPPs) are characterized by long term relationships often around multiple projects and producing work that is mutually beneficial [3].In RPP's, researchers and practitioners work together to identify problems that are situated in practice and develop research designs to inform practice-based interventions.The work is conducted through a model of shared expertise and shared authority.We use examples from our own research on learning in a physics-focused interactive science center conducted through a Research-Practice Partnership and describe lessons learned through research about establishing RPPs and methodological considerations when conducting research in informal science education settings such as interactive science centers.

II. ESTABLISHING PROBLEMS OF PRACTICE
The work described in this paper was conducted through a research-practice partnership between MOXI, The Wolf Museum of Exploration + Innovation and the University of California at Santa Barbara (UCSB).MOXI is a new interactive science center for visitors of all ages.MOXI was being designed at the time that the Next Generation Science Standards (NGSS) [4] were released, providing the opportunity to structure the museum experience to complement the school-based The geographic proximity and the consistent educational stances of UCSB's Education Department and MOXI form the basis for mutually beneficial collaborative work.Since opening in 2017, we have collaborated through an RPP model which began by identifying the needs of MOXI and UCSB and co-developing mutual problems of practice and co-designing curricular materials and programs, which in turn, led to specific researchable questions.
As described above RPPs work together to identify problems of practice.The problems of practice and research questions appropriate for work in museums differ from other educational contexts because the ways museum learning environments differ from classrooms in the learners, educators, participation structures, and outcome goals [8].Unlike classrooms, in which the learners are typically students of one grade level, allowing instructors to make some assumptions about students' prior ideas, the population of learners in a museum is the general public; children and adults may interact with an exhibit simultaneously and have a range of motivations for visiting [9].The population of museum educators also differs from that of classroom educators.While classroom educators typically have a degree in their field and perhaps many years of teaching experience, museum educators are often temporary part-time employees who are novice teachers and have considerably less content knowledge.The activities associated with teaching and learning vary in their repetition and duration.Classroom educators have repeated opportunities to work with the same students for predictable amounts of time (e.g., a class period) and can prepare lessons in advance.In contrast, museum facilitators might see a visitor once and for only few minutes; however, they have the opportunity to work with the same content many times a day, facilitating the same exhibit repeatedly with new visitors, potentially trying out new ways to facilitate with each iteration.
Our work is further informed by prior research in museum visitors' experience [10], theoretical framing of constructivism [11][12][13][14] and identity work [15,16], and MOXI's mission.Together, these areas influenced our decision to develop learning opportunities and frameworks to increase opportunities for visitors to engage in practices aligned with STEM disciplines (e.g., asking questions, developing explanations) [17].These practices are informed by the NGSS [4] and our observations.We expect that by engaging in STEM practices visitors will construct understanding of STEM topics and develop these practices as tools for thinking about the world, leading to positive identities as STEM learners.
Such learning experiences require appropriate facilitation.Despite consensus that student reasoning should inform instruction and robust models to support such instruction in classrooms [18][19][20], it remains difficult to implement.In fact, despite museum leadership expressing goals of increasing visitor-centered participatory experiences, didactic instruction is more common in museums, even after training [21].This is not surprising.Facilitating learning in ways that support visitors in constructing their own understanding is difficult, especially since facilitators may be working simultaneously with children and adults of a range of ages, backgrounds, and goals.Furthermore, the museum staff who facilitate the learning experience of visitors are often part-time staff, lacking a background in science and engineering, with few opportunities for training.Thus, we are also concerned with helping facilitators develop appropriate skills, leading to two primary areas that guide the direction of most of our work: Practice-based Learning and Practice-based Facilitation.Practice-based Learning refers to learning experiences in which visitors engage in learning through STEM practices.We refer to the facilitation process that allows for such learning opportunities as Practice-based Facilitation.

III. RESEARCH AND PRACTICE DESIGN AND RESULTS
Our work follows a Design-Based Implementation Research (DBIR) approach.DBIR [22] includes four principles: 1) focus on persistent problems of practice from multiple stakeholders' perspectives; 2) commitment to iterative, collaborative design; 3) concern with developing theory related to both school (or museum) learning and implementation through systematic inquiry; and 4) concern with developing capacity for sustaining change in systems.Our work is designed to understand practice-based learning and practicebased facilitation.The specific methods of data collection and analysis vary according to the specific research questions.As described above, RPPs are characterized by multiple interacting programs.We currently have two long term programs and additional smaller programs.Below we describe the two larger programs: a program to train facilitators and a field trip program.Both of these programs began without funding and now funded by the National Science Foundation (NSF).Following that, we discuss smaller projects and, finally, how these projects incubate additional partnerships.
Facilitator training.Our research on practice-based learning began by investigating visitors' experiences.Initially, we sought to understand the ways visitors engaged with open-ended exhibits [23][24][25] and how these aligned with our STEM practice goals.Our second focus of practice-based facilitation guided our analysis of the visitor experience to inform strategies for making facilitation decisions that would support visitors in engaging in STEM practices [17].That is, our practice-based goals prompted us to code in ways that would inform tools to improve facilitation practices.This resulted in a framework for facilitation that included specific facilitation moves.Specifically, for target exhibits, we identified engagement levels or common ways of engaging with the exhibit that a facilitator can easily identify through observation and then linked these observation levels to practices that visitors might be engaging in.For example, at one exhibit, visitors use a wall full of pegs and rubber tubing to construct ball roller coasters.A visitor who is repeatedly rolling a ball down a track and changing the release height each time may be making observations and testing a variable (two STEM practices).After observing the visitor, a facilitator might choose to ask a question or pose a challenge that prompts the visitor to change the way they are engaging (e.g., prompts them to adjust the track), engage in practices that they are not already engaging in, or use a specific practice in a more sophisticated way (see [17] for more details).
This research on visitor learning at exhibits informed the development of a year-long training program for informal science educators designed to increase the stability of MOXI's floor staff [26] Coordinated field trip and classroom activities.Our second area of research and practice relates to MOXI's field trip program [27].We identified a need among local elementary school teachers to develop skills in teaching NGSS-aligned science and engineering and recognized that MOXI's field trip programs could be leveraged toward addressing this goal.Engineering Explorations is a 3-year program funded by the National Science Foundation (NSF) with goals of developing modules that consist of MOXI field trip programs and coordi-nated activities implemented in elementary school classrooms before and after the field trip.Our goals include both student learning about science and engineering and teacher learning and confidence about implementing engineering instruction.Research on this program is in early stages.
Activity prototyping.The partnership has provided a context for graduate students to test programs in MOXI that are later used in classrooms.For example, one student developed a Virtual Reality program for children to learn coding [28].Another designed and implemented an activity for middle school students diagnosed with ADHD to design and fabricate fidget tools [29].Testing in MOXI provides an the opportunity to repeat programs with many different guests allowing for rapid iteration, and the informal context allows for lower stakes environments to test activities to be used in other contexts.These research projects have informed dissertations and other programs beyond MOXI.
Incubating new partnerships The UCSB-MOXI partnership has developed and implemented programs that collectively serve museum educators, museum visitors, elementary school teachers, elementary school students, after school programs, undergraduate researchers, undergraduate prospective teachers, community college students, high school teacher credential candidates, and graduate students.We value the shared expertise and see evidence that the collaboration improves the work of both the practice and research.We use our experience and programs to incubate new partnerships among early career scholars and practitioners.The programs developed through this collaboration bring together novice researchers (graduate students and undergraduate research assistants) and novice practitioners (beginning museum educators) to conduct research and engage in practice together.Some courses designed for the certificate program are crosslisted with UCSB graduate courses for doctoral students, creating opportunities for graduate students and MOXI facilitators to work together to conduct research.In the Spring of 2018, four teams of novice researchers and practitioners conducted focused studies on MOXI exhibits.This work furthered our research and provided opportunities for novice practitioners from MOXI and novice researchers from UCSB to collaborate [24].Similarly, Engineering Explorations includes UCSB undergraduate researchers and graduate students who collect and analyze data.They develop research questions that stem from the data collected and contribute to the program goals, directly experiencing how research and practice inform one another.These intentional collaborations foster productive mindsets toward conducting research and practice through Research-Practice Partnerships from the beginning of the their careers.

IV. LESSONS LEARNED -RESEARCH DESIGN
In this section, we present what we have learned from our work conducting research in a museum space to help others embarking on similar research.

Agreeing on Problems of Practice
Agreeing on what to investigate can be a source of tension.Conducting rigorous research may require long term extensive data collection and longer periods of analysis.This may be at odds with the goals of museum staff who want to know what to do the following week.Developing research plans that can simultaneously support rigorous research and produce quick findings that can inform museum operations may require collecting and analyzing multiple smaller batches rather than waiting until all data are collected to begin analysis.The interim analyses can inform practical operations as research is continuing.
Video data collection.Our work includes video-based data collection, observations, and surveys.For video-based observations, we have used security footage, point of view cameras, handheld cameras/smartphones, and cameras on tripods.Here, we focus only on the first two since the others are also commonly used in classroom data collection.The footage from the security cameras does not include audio recordings and the (immobile) cameras are positioned to capture as much of the floor as possible.Video is continuously passively recorded and can be watched on high speed.These last two aspects allow us to efficiently investigate certain types of questions such as how long visitors stay at exhibits, what sorts of paths they take through the space, and whether people explore exhibit in groups or alone.However, the lack of sound recording limits its application.The resulting data is not appropriate for investigating visitors' developing understanding.Further, according to our human subjects board approved protocol, this video cannot be shown at conferences or analyzed outside of MOXI.Point of view cameras (small cameras that are worn on a visitor's or facilitator's head or shoulder) [30] record video and audio from a specific individual's perspective, allowing data to be collected on what visitors or facilitators notice, how visitors use an exhibit and interactions between visitors and facilitators.
IRB and participant consent.MOXI, like many museums, does not have its own IRB.Thus, all MOXI studies are reviewed by UCSB's Internal Review Board (IRB) for Research on Human Subjects.Collaborators at MOXI complete UCSB's online IRB training module.Our methods of data collection and process for collecting informed consent and parental permission vary considerably across the studies.For studies of students attending field trips, we work with the participating schools to send home parental permission forms for research participation.We request teachers collect these forms.As per our IRB requirements, museum staff solicit parental permission through schools.This is so that researchers do not acquire parent or teacher contact information.For children attending summer camp, we send information on the research project and parental permission forms home with families prior to camp or provide the permission slips to parents on the first day of camp.For activities that take place in a specified area for a limited time, we use stanchions to section off an area and ask visitors that match target demographics (e.g., specific age) if they want to participate in the research.If so, they sign a consent form and partic- ipate in the activity.Visitors who do not want to participate are neither filmed nor interviewed but do participate in the experience.For research on how visitors typically interact with exhibits and facilitators, we place a sign on the front desk that alerts visitors that filming is occurring in the museum.This sign specifies where filming is taking place and requests that visitors who do not want to be filmed wear a colored wrist band to alert researches that they do not want to be filmed.At the filmed exhibit, another sign alerts visitors of filming in progress.If visitors who do not want to be filmed are in the area, the cameras are turned off.We do not conduct videobased data collection during times that school field trips are in the museum to avoid filming children who are not accompanied by adult guardians.

V. LESSONS LEARNED -ESTABLISHING MUTUALLY BENEFICIAL PARTNERSHIPS
Sustainable partnerships must benefit both partners, which requires work.Our partnership benefits MOXI through developing and supporting programs and conducting research in areas that align with their mission.UCSB benefits by gaining research contexts for students and faculty.Both benefit from the expertise of the other.The recommendations below are for researchers establishing partnerships with museums.
Be involved.It is important for researchers to know what is going on in the museum beyond just the research.Understanding the variety of learning opportunities helps researchers understand the opportunities for learning.It is also critical to understand the museum's vision for the visitor experience.By working with museum visitors, the researcher becomes a part of the museum experience and the research should not detract from the museum experience.
Write together.Research points to difficulty in RPPs communication because they lack a shared language [31,32].Coauthoring papers and grant proposals requires articulating intended practice and research deliverables and planning the research methods.This process of putting ideas into words surfaces differences in goals and understandings that need to be worked out.Similarly co-authoring papers ensures partners are interpreting data in the same way.
Share expertise.While collaborative writing is useful, partners will have different strengths.Researchers provide expertise in research methods and access to university libraries and research literature.Practitioners have expertise in how the research methods would work in the museum and what problems of practice they encounter.
Share authority.RPPs are characterized by shared authority.One way our work demonstrates this is through collaborative grants.For many divisions, NSF allows for institutions to submit separate but connected proposals.We submit one shared project description and separate budgets.This results in two grants, one to each organization so that each institution controls their own funding stream and each institution has a PI.However, projects with multiple organizations and multiple interrelated projects require complex project management and identifying who will be involved in decision making.Identifying and using a project management model (e.g., RACI [33]) across the RPP will facilitate this process.

VI. CONCLUSION
Physics Education Research (PER) has long been focused on producing evidence-focused practice.Work in PER sometimes follows a model of a practitioners (physics professors) collecting evidence on their own classrooms to understand what works and what does not and to produce knowledge and theory that contributes to a larger literature base, evidence of deeply valuing the expertise of practitioners.Likewise, Research-Practice Partnerships (RPPs) [1] [2] highlight the important role and expertise of practitioners in research and the important role research can play in practice.In our in work in informal physics education, we have found that establishing a Research-Practice Partnership and working to develop research programs around problems of practice has benefited both organizations and led to rich research contexts.RPP models of collaboration are also used in work with K-12 schools and districts.Partnering with classroom teachers through RPPs provides opportunities for teachers to contribute to the research process by drawing on their experience in the classroom being studies and steering the research towards problems of practice.Research that focuses directly on problems of practice experienced by the practitioner, in formal or informal settings, results in research outcomes that are directly usable by the practitioner and contributes to the sustainability and investment of the practitioner in the research.

FIG. 1 .
FIG. 1. MOXI, The Wolf Museum of Exploration + Innovation . The participants of this training program enter as a cohort and go through the program together, culminating in a professional Certificate in Informal STEM Learning, awarded through UCSB.The program has two interrelated components: (1) Part-time paid work as a MOXI facilitator and (2) Coursework directly tied to their work at MOXI.The participants interact with visitors by facilitating exhibits, demonstrations and activities at program carts, and educational programs.They also work with students in schools during outreach programs and field trip programs (see description of Engineering Explorations below), engage in curriculum development, and conduct research in MOXI.Starting with our third cohort, which began in Fall 2019, we are conducting more systematic research, funded by the National Science Foundation (NSF).

FIG. 2 .
FIG. 2. Notice of Filming at exhibit (left) and sample text of sign at front desk (right)