The effect of explicit preparation in pedagogical modes on informal physics educators

Participation in informal education programs has been demonstrated to increase volunteers’ ability to communicate science and to affect their motivation for continued participation. Recently, it was reported that volunteers tend to engage with students in one of three distinct pedagogical modes, of which they may or may not be aware. As a preliminary step in the investigation of these modes and their effect on instructor manner and efficacy, we have implemented a module on the modes in the basic training given to all volunteers of the Partnerships for Informal Science Education in the Community program prior to the semester. Using preand post-surveys, we have characterized the shifts in their perception of the modes and compared them to semesters prior to the implementation of the training. We find a significant, immediate shift in volunteers’ perceptions and here detail these findings as well as potential pathways for further study.


I. INTRODUCTION
Physics Outreach can be defined broadly as a set of programs, initiatives, and activities designed to increase public awareness of physics [1] in an informal setting such as a museum or after-school program [2,3].Such initiatives can also have an affect on the attitudes and dispositions of participants, making outreach useful not only for communicating science to the public but for increasing scientific affect as well [4,5].Physics outreach initiatives can (and, some would argue, should, see e.g.Refs.[6,7]) also be beneficial for the "experts" nominally "performing" the outreach.Several studies have assessed various aspects of informal education programs staffed by university physics students, with various beneficial findings [8][9][10].
Most recently, it was reported that student University Educators (UEs) engage in specific modes of interaction when speaking with non-experts within the context of an afterschool informal science program.As described in Ref. [11], a socio-cultural view of student activity was utilized along with a cultural-historical activity theoretic framework [12,13] to characterize UE pedagogical behavior according to three modes: instruction, in which UEs are positioned as authority figures over students and impart content knowledge, consultation, in which UEs are positioned as advisors to students and lend advice or praise as requested, and participation,, in which UEs are positioned as co-participants alongside the students and share agency with them in the experiment.These modes are additionally described in the activity theoretic framework by the various and particular objectives, roles, and rules of engagement that characterize the interactions between the UEs and the students.In addition, it was reported in Ref. [11] that, without necessarily being explicitly aware of the existence of the modes, UEs tended to favor the consultation mode over the instruction and participation modes by about a factor of three.
Building upon the work reported in Ref. [11], we examine UEs' disposition -that is, their attitudes toward and preferences between each of the modes of interaction presented in that work -before and after an intervention that included explicit examination of the modes, with the goal of determining the effect of exposure to the modes on the UEs' preferences between them, in accordance with the recommendations from Ref. [11].We performed this study in the context of the Partnerships for Informal Science Education in the Community, an after-school program based at University of Colorado Boulder (CU) designed to engage underrepresented K-8 students in local primary schools [14].In this work we briefly discuss PISEC and UE training procedure and present data suggesting that simply making UEs aware of the modes of interaction presented in Ref. [11] can have an impact on their disposition toward the modes within the context of informal education.

II. BRIEF DISCUSSION OF PISEC AND UE TRAINING
The Partnerships for Informal Science Education in the Community is a program that fosters a series of long-term educational partnerships with local community sites of education.PISEC is jointly supported by the JILA AMO Physics Frontier Center [15] and the Department of Physics at University of Colorado Boulder and partner sites are typically public schools in the local community, often with higher percentages of underrepresented minority students or students of lower socioeconomic status.One of the overarching goals of PISEC is to provide pathways for students to participate in the science community (and physics community in particular) while practicing the skills professional members of the community use on a daily basis, thereby making these communities (and their affiliated careers) more accessible.
PISEC, which is based on the Fifth Dimension (5D) model of informal education [6], typically takes the form of a onehour afternoon session at the partner site where groups of 3-5 students explore inquiry-based science activities from various physics-based curricula, assisted by UEs who work as nominal co-participants with students and facilitate the activities.The activities and program are designed to facilitate the development of science identity in primary students

PERC Proceedings,
even as it allows UEs the opportunity to develop identities as part of a physics culture that embraces diversity and outreach.The UEs that participate in PISEC are typically undergraduate or graduate students from JILA or the CU physics department, although UEs from other departments or at other stages of their academic careers also occasionally participate in the program.UEs volunteer for one of 3-5 PISEC site for an entire semester and make 8-10 weekly visits throughout the semester.UEs typically express multiple and varied reasons for PISEC participation, ranging from simple curiosity of CU outreach programs to the interest in developing teaching skills to the desire to give back to the community.
Prior to the first visits of the semester to PISEC partner site, UEs undergo preparation in the basics of informal education and the particulars of PISEC's implementation.The preparation sessions include three modules: curriculum, communications, and diversity/pedagogy.PISEC visits multiple sites per semester, each implementing a separate curriculum of physics activities; in the curriculum module, UEs practice the activities they will facilitate at their specific PISEC site.Because UEs are working with young non-experts, they practice scientific communication through the creation of short movies explaining scientific concepts at an elementary-school level in the communications module.Finally, in the pedagogy/diversity module, UEs are made aware of the specific demographic particularities of each of their sites, engage in activities designed to expose them to the idea that students of a lower socio-economic status are not inherently less creative or inquisitive than other students, and learn about PISEC's overall teaching philosophy and the inquiry-based paradigm.
In the fall semester of 2016, the pedagogy/diversity module was updated to include more discussion, engagement, and reflection on the topic of diversity and to reflect and to incorporate the findings of Ref. [11] in the form of a purposeful intervention to assess UEs' dispostion toward the modes.We will not here report the changes made to address issues of diversity and inclusion, instead focusing on the changes made to introduce UEs to the framework of pedagogical modes.During the module, UEs were exposed explicitly to each of the modes, first through what essentially amounted to summaries of the salient traits of the modes as presented in Ref. [11].Several slides were shown giving descriptions, definitions, and traits of the modes, in the order in which they are presented in Ref. [11] (instruction -consultation -participation).UEs were then given a chance to practice each of the modes in a roleplay scenario wherein they took turns acting as the mentor to other UEs acting as students running through successive instances of a single PISEC activity.It was made explicit to the UEs multiple times throughout the course of the module that the modes were not being presented in any sort of hierarchy, or with any kind of normative expectation.In addition, it was not shared with the UEs that consultation mode was found most prevalent among those surveyed in Ref. [11], so as not to prompt any particular response from the UEs.

III. METHODS
In previous semesters of PISEC, UEs have been given pre-and post-semester surveys charting their attitudes toward PISEC participation, informal education's place in physics, teaching styles, etc.The pre-semester surveys are typically given before training, or at least before the Pedagogy/Diversity Workshop, so as to get UEs' natural opinions.As in previous semesters, we administered a pre-semester attitude survey.However, as part of the modes training intervention, we also administered a separate survey after completion of training.Using the pre-and post-training surveys, we measured the effect of explicit training in the modes on UEs' attitudes toward teaching styles.We analyzed only those students who responded to both the pre-and post-training surveys, n = 12.In this population of UEs, the male-to-female ratio was 3:1 and no racial or ethnic background or demographic information was collected.We also administered a post-semester survey at the conclusion of the PISEC semester to determine whether any shifts observed in the post-training survey had persisted throughout the semester.
The pre-training survey was identical to the pre-semester survey given in previous semesters and included the question "What is your philosophy/approach/style of teaching?Please explain why you have chosen this philosophy."(this question appears on both the pre-semester survey and the postsemester survey, given to UEs at the end of each semester) No mention of the modes was given at this point.Using the definitions of the modes, we developed codes for each of the modes.Analyzing UE responses for key words and themes, we inferred preferred modes from responses.In general responses could be categorized relatively easily based on the tone of the response and the words used.For example, one UE's response to the question: "I believe that learning should be a self motivated process.My purpose as an educator is to mediate and encourage this motivation.Force feeding facts and equations will not result in any meaningful growth as a scientist or individual."We interpreted this answer as evidence of a primarily consultation-oriented teaching preference, based on the distinct roles implied in the response, the description of students' learning as a "self motivated" process, and the apparent preference against "force-feeding facts and equations," which might be expected for instructionoriented teaching.We ensured reliability by having two raters code UE interviews independently, finding good agreement between the independent results.
The post-training survey (again, new for the Fall 2016 semester) focused on the Pedagogy/Diversity workshop and included the question "Based on your own understanding of the the three modes presented in the Diversity/Pedagogy Workshop and your own approach to teaching (not necessarily in agreement with the presentation), which mode of the three presented (Instruction, Consultation, Participation) do you think is the most useful?And why?"The parenthetical about agreement with the presentation was included to reinforce our attempt to not present the modes in any hierarchical fashion.The results of both the pre-and post-training surveys are reported in Figure 1.

IV. RESULTS AND DISCUSSION
As shown in Figure 1, it appears that explicit training in the pedagogical modes of Ref. [11] has an immediate effect on UEs' preferences between modes.In the pre-semester survey, all but one of the twelve UEs analyzed expressed a preference for the consultation mode.One UE used language that could be seen as potential evidence for multiple preferences, both consultation and participation modes (the UE's expressed preference for "walking alongside [students] as they learn and explore to help them come to satisfying conclusions and insightful connections" could imply shared roles as participants and learners).For the single UE who was inferred to have expressed preference for instruction mode, the inference was made based on the fact that the UE described student understanding as a result of his own actions and spoke from a position of separate roles ("... when you build off of different topics and relate them together students understand the relations and can perform multiple step problems correctly.

" [emphasis added]).
The results of the post-training survey, however, demonstrate not only a general shift in UE teaching preference from consultation mode to participation mode but a greater breadth of consideration concerning the modes.Of the twelve UEs surveyed, nine expressed a preference for participation mode either solo or in conjunction with other modes, and the three who did not maintained their stated preference for consultation mode.Several of the UEs mentioned multiple modes in their responses; however, we only included in Figure . 1 those explicitly-mentioned modes that UEs described as being superlatively useful (e.g. using "most helpful" or simi- lar language).As reported, two UEs expressed equal preferences for both participation and consultation modes, and a single UE expressed equal preferences for all three modes ("I don't see one as more useful than the others.Each has its own strengths and weaknesses...").All three of these UEs expressed a preference for consultation mode in the presemester survey.Interestingly, the single UE who expressed a preference for instruction mode in the pre-semester survey underwent a seemingly-dramatic shift to participation mode in the post-training survey.
In order to assess the likelihood that the observed shifts were the result of statistical fluctuations, we applied the Bhapkar test [16], which measures the marginal homogeneity of the data (that is, the test measures the likelihood that observed shifts in pre-and post-intervention preferences are due to the intervention itself and not statistical fluctuations), to the data reported in Fig. 1 to determine a test statistic χ 2 and pvalue.We calculated a test statistic of χ 2 = 35.86,(ν = 4) with corresponding p < 0.001, indicating an extremely high likelihood that the observed shift in UE attitudes was due to exposure to the modes during training.
In order to benchmark the shift we observed as a result of training in Fall 2016, we analyzed pre-and post-semester survey responses from three previous semesters: Fall 2012, Spring 2014, and Fall 2014.We again only analyzed responses from UEs who had responded to the question "What is your philosophy/approach/style of teaching?Please explain why you have chosen this philosophy."on both the presemester and post-semester surveys.Throughout the three semesters, we analyzed a total of 13 UE responses.We analyzed both pre-and post-semester responses in the same manner as we analyzed the pre-semester responses from Fall 2016; that is, by using definitions from Ref. [11] as the basis to infer mode preferences.The pre-and post-semester surveys are shown in Figure 2, split by semester.As shown, there is an overall slight decrease in all three semesters in instruction mode preference and a slight increase in participation mode preference, with most UEs favoring consultation mode.In concert with our statistical test, these findings lend credence to the idea that the immediate and sizable shift after explicit exposure to the modes in training appears to be a result of the training itself.The data in Fig. 2 was compiled from every UE who completed both the pre-semester survey and the post-semester survey.
It is apparent from these findings that even simply exposing the UEs to the modes of interaction formulated by Hinko et al. [11] has the effect of increasing the breadth of their responses, potentially indicating a propensity for more nuanced thought about the applicability of different pedagogical techniques.Our findings also suggest that training in the modes of interaction in the context of PISEC produces a shift in disposition toward a more active, participatory mindset in UEs within the PISEC setting.Interestingly, a majority of UEs explicitly qualified their preferences as pertaining to PISEC itself, using phrases such as "... [participation mode] seems the most in line with the goals of PISEC," or "In the case of PISEC, I find ..." This could indicate that the UEs do not perceive the educational values of PISEC (collaborative work, "doing" being more important than "knowing," relaxed and student-led structure) to be commensurate with the values they may perceive to be part of a formal learning environment, where practices and structures tend to more nearly often an instructional model.

V. CONCLUSIONS AND FURTHER STUDY
We have shown that simply exposing UEs to the pedagogical modes reported in Ref. [11] has an immediate, significant effect on their attitudes and preferences for the modes after completing training.Preliminary analysis, however, suggests that, though preparation may produce the observed shift in the context of training, the practice of teaching in PISEC may cause UEs to modify their preferences.The results of the post-semester Fall 2016 survey, identical to the post-semester surveys used in past semesters, appear to indicate that by the end of the semester, UEs' pedagogical mode preferences were similar to those observed in previous semesters (namely, a larger preference for consultation).The extent and cause of this shift are currently under continued investigation; for example, it could be that UEs lack the resources to enact a more heavily participatory pedagogy, or that the PISEC environment implicitly promotes a consultation mode, essentially influencing beliefs and practices more than the training does.A similar finding was reported for formal settings [17].Thus, it may be pertinent to consider revising the training paradigm to reinforce PISEC perspective and support UEs as they navigate its implementation.
Pertinent to uncovering the answers to questions raised by these findings, of course, is the question of how UEs actually teach children in PISEC following this updated training.We recorded UE/student interactions throughout the semester and intend to follow up with several of the UEs, including those who exhibited shifts between pre-and post-training surveys.In addition, the data we collect should allow for future improvements to training in PISEC's informal pedagogical paradigm and potentially allow for broader and more nuanced training during university students' formal TA training as well.
edited by Ding, Traxler, and Cao; Peer-reviewed, doi:10.1119/perc.2017.pr.008Published by the American Association of Physics Teachers under a Creative Commons Attribution 3.0 license.Further distribution must maintain attribution to the article's authors, title, proceedings citation, and DOI.