Participants’ perceptions of the Faculty Online Learning Community (FOLC) experience

Melissa H. Dancy,1 Alexandra Lau,1 Joel C. Corbo,2 Charles Henderson,3 and Andy Rundquist4 Department of Physics, University of Colorado, 390 UCB, Boulder, CO 80309 Center for STEM Learning, University of Colorado, 393 UCB, Boulder, CO 80309 Department of Physics, Western Michigan University, 1903 W. Michigan Ave., Kalamazoo, MI 49008 Department of Physics, Hamline University, MS-C1913, 1536 Hewitt Ave., St. Paul, MN 55104


I. INTRODUCTION
The Faculty Online Learning Community (FOLC) model is designed to provide a support mechanism for faculty who attend the multi-day Physics and Astronomy New Faculty Workshop [1,NFW].Previous work demonstrates that the NFW is highly successful at increasing awareness of Research Based Instructional Strategies (RBISs) among faculty participants and motivating them to transform their teaching [2].However, evidence indicates that knowing about innovations and being motivated to try them are frequently not enough to bring about sustained and impactful change [2].Longitudinal data from the NFW indicates that, while participants are significantly more likely to report knowing about and having tried RBISs than faculty who did not attend the workshop, they are no more likely to report sustained or high level use [2].This is because faculty members who try an RBIS typically run into implementation difficulties that they need ongoing support to work through [3,4].The FOLC, inspired by in person Faculty Learning Communities, was designed and implemented to address this challenge [5].In this paper, we report on efforts to better understand how the FOLC addresses this challenge by considering participants' self-reports of their motivation to join and impacts of the FOLC experience.
A FOLC typically consists of six to ten participants and two facilitators.Participants in a given FOLC all attend the same NFW and are recruited through both emails and announcements at the NFW.An evening is set aside at the NFW for potential participants to meet each other and the facilitators of their cohort's FOLC.Once formed, the FOLC is officially active for about a year.During this time, participants and facilitators meet synchronously via video conference every other week and communicate asynchronously via an online message board system between meetings.Synchronous meetings consist of a mix of guest speakers sharing how they implement a particular RBIS and time for discussion among the participants about their own teaching practices and challenges.The topic of each meeting is determined based on requests from the participants.During the second half of the year, participants are expected to engage in scholarship of teaching and learning (SoTL) projects, shifting the emphasis of the participants from learning from external experts to implementing and reflecting on their own reforms and providing feedback to each other [6].
In previous work we reported on the design principles of the FOLC [7] and initial results from our pilot cohort [8].We are now two years into the project and have three completed FOLC cohorts.This paper discusses results from these first three cohorts, which indicate that the FOLC model is successful at increasing participants' confidence in trying new teaching techniques, promoting reflective practice, and providing additional knowledge of the different RBISs.

II. MODELS FOR EDUCATIONAL CHANGE
All change efforts are designed around (sometimes implicit) models of how change works; whether a particular change effort is likely to succeed depends in part on level of alignment between the model and the details of the change.The common Development and Dissemination (D&D) model of change focuses on the transmission of knowledge about innovations whose development is finished; it assumes that once an innovation has been "proven to work," all that is required to ensure wide adoption is to spread the word about the innovation.However, this model typically fails at spreading educational innovations because it fails to account for the systemic, environmental, or cultural barriers present in educational settings (e.g., inability of faculty to control all aspects of their classroom, inapplicability of innovations developed in one context to radically different ones, departments unsupportive of educational innovation).If these barriers are present in a department, they will prevent all but the most dedicated (or stubborn) of faculty members from implementing even the best developed and tested innovations.
Implicitly, the NFW uses the D&D model of change.The NFW provides participants with an intensive, multi-day exposure to many teaching innovations, often presented by noted developers or researchers of those innovations.Participants leave the NFW excited to try these new innovations, but they may only have a surface level understanding of how to implement them.When they inevitably run into challenges, the barriers discussed above will tend to make them abandon the change unless there are support structures in place to counteract this effect.
The FOLC was designed to supplement the NFW's D&D change model through a community-based change model.This kind of change model assumes that a faculty member trying to implement an educational innovation will need support to do so, in the form of a community of peers that can help them overcome barriers.This community could be their own department.When that isn't the case, some other group can step in to serve this function, thereby enhancing the chances of the faculty member sustaining their change.
The FOLC was created to serve as this supportive community for its participants.We hypothesize that when participants encounter barriers to implementing teaching changes, instead of abandoning the changes they will turn to their FOLC peers and facilitators to (1) learn more about the change they are trying to implement, (2) gather feedback from the group and troubleshoot the problems they are having, and (3) receive encouragement to push through with their changes.We further hypothesize that participants in the FOLC will become more reflective about their teaching, both in realizing when changes need to be made and when deciding what new strategies will work for their class.This capacity for reflective teaching is critical in order to modify a teaching innovation designed for one context to a different one, which is a skill not addressed in D&D approaches to change.

III. COHORTS, DATA SOURCES, AND METHODS
To date, three cohorts have completed a FOLC.The first cohort ran for one semester (these participants did not conduct a SoTL project), and all the following cohorts have run for a full year.Participants were in their first few years of a faculty appointment in a physics or astronomy department.Table I indicates the breakdown of each cohort by gender and by institution type.The FOLC participants were more likely to be female than the general NFW participants (50% in FOLC vs. 30% in NFW) and more likely to be from an undergraduate-only department (70% in FOLC vs. 55% in NFW) [9].This is a self-selected group of participants and they are not representative of physics faculty overall.Cohort 3 ended up smaller than our other cohorts because three people dropped out after the first semester.
For every cohort, we collected recordings of the synchronous meetings and extracted a log of all posts from our asynchronous platform; an analysis of asynchronous interactions is reported elsewhere [8].We also conducted interviews with FOLC participants, which we report on here.For Cohort 1, these interviews occurred approximately midway through their time in the FOLC.Because it was our first cohort, we were running it as a one-semester pilot, and we wanted to get feedback from participants before it was over.We conducted post-interviews with Cohort 2 and 3 participants after they completed their year in the FOLC.All participants in Cohorts 1 and 2 were interviewed.However, of the seven participants in Cohort 3 who completed the FOLC, we were only able to interview five of them, and of the three that dropped out we were only able to interview one.Data from the Cohort 3 dropout is included in data reported below of reasons why participants joined the FOLC, but we only report impact data from people who completed their FOLC.We used a semi-structured interview protocol.Participants were asked: (1) general questions about how the FOLC went, (2) about their motivation for joining the FOLC, (3) whether they found the experience worthwhile, (4) how the FOLC impacted their teaching, (5) about their impressions of the community that developed among their cohort, and ( 6) specific questions about the different programmatic components of the FOLC: synchronous meetings, asynchronous communication, and SoTL projects (if applicable).
Two of the authors (MHD and AL) coded the interviews.As a first pass, all the responses to the question about motivation for joining the FOLC were coded as "Motivation," and any time the participant spoke about how the FOLC has impacted them was coded as "Impact."We then developed (separate) sub-coding schemes for the Motivation and Impact codes.Some of the categories of the sub-coding scheme were developed based on patterns we had seen in our first pass at coding, while others emerged as we went through the responses a second time.The two authors sub-coded all the entries separately and then compared their coding until total agreement was reached.

IV. RESULTS
In this paper, we report only on participants' self-described motivations for joining the FOLC and immediate impacts of the FOLC on them.Other forms of data, including longitudinal data, will continue to be collected and analyzed for a more complete picture of FOLC success for later reporting.

A. Motivation for Joining the FOLC
Participants' reasons for joining the FOLC matched our expectations and goals for the FOLC in that they were looking for a community and not just signing up for more presentations from experts.Of our 24 interviewees, half reported that one of their motivations for joining the FOLC was to have implementation help as they tried out new teaching strategies.Moreover, 80% of interviewees reported that one of their reasons for joining was to gain a community of supportive peers with whom they could discuss their teaching.Of the twenty people who stated this motivation, nine further specified that one of the reasons they wanted such a community was because their department was unable to provide that support.For example, a participant explains, "I feel like our department specifically has gotten a little old fashioned, if you will, and so I was excited to kind of think about some new strategies that could be utilized in physics."This person didn't have many people to talk to in their home department about non-traditional (research-based) teaching strategies.Another participant says, "So having faculty who like to try the different kinds of techniques and trying them out and seeing what works is not the easiest to find at a very, very small school."A number of our participants came from small departments (even one-person departments) where they did not have many (or any) people to talk to about physics teaching.

B. Impact of the FOLC
Almost all of the participants interviewed (22/23) had positive perceptions of the FOLC: they felt that it was worth their time, would recommend it to future NFW participants, and/or had something strongly positive to say about some aspect of the FOLC.The remaining participant was neutral in his perceptions.
Participants reported being impacted by the FOLC in a number of ways (see Table II, and bold text below).Most participants described trying new teaching strategies during their time in the FOLC or plans to try new techniques.The few who did not report this impact were all part of the shorter pilot cohort (so they did not have as much time to try something new as those in Cohorts 2 and 3) and two were not teaching the semester they were in the FOLC.An equal number of participants talked about gaining knowledge during their FOLC, both from guest speakers and from feedback from other cohort members.This knowledge was not limited to teaching techniques and materials; some participants talked about how it was nice to learn how other departments did things and what other schools were like.
A complete examination of these two impact categories will be included in a forthcoming paper, but here we will focus on two of the other impacts reported by participants: increased reflection and confidence.We note that there was no interview question that specifically asked about reflection or confidence so the numbers reported in Table II represent those who spontaneously reported these impacts.We suspect that additional FOLC participants would have agreed that they experienced these impacts had we asked them directly.
Nearly half of the interviewed participants talked about Other participants talked about how the project they did in the second half of their FOLC led them to reflect on their teaching.One participant says, "Rather than just having this idea that I want to try, well that's good, but let's think about what the goal is of this new attempt and how to measure whether it's working or not so that you know whether it's been successful, whether it should be continued or whether it needs to be modified.So I think that it was a good experience to be forced to stop and think about that before jumping into something." One-quarter of the interviewed participants said the FOLC helped increase their confidence in attempting new RBISs.An illustrative example of this confidence is when one participant describes building her syllabi for the new quarter and explains, "I guess I feel more confident putting things into [the syllabus] that I might not have otherwise because I know I can go back to folks and say okay, I'm trying to do this, tried it in the first lecture and it didn't work, what do you suggest?"She describes her cohort being a sort of "safety-net" to help her troubleshoot any issues she has, and that has encouraged her to try new teaching strategies.Another participant explains how his gain in confidence helped him persist with a teaching change.He states, "I feel more like everyone's in the same boat, not like oh my god, I don't know what I'm doing, I don't know how to implement this thing correctly.Okay, no, here's a bunch of people who are all in the same boat and we're all fumbling around trying to figure out how to do this the best, and this is not unique to me.That made me feel more confident about okay, if I don't feel like I'm doing 100 percent, it's the same as everyone else and I should just keep on iterating and figure out how to do it better."Confidence was gained not only because of the support given by the cohort, but also because of the comfort in the knowledge that everyone encounters implementation issues.

V. DISCUSSION
As noted previously, the D&D model of change is insufficient as it does not consistently lead to sustained change.The FOLC model is designed to promote sustained change by supporting faculty over an extended time as they navigate the sometimes difficult process of integrating more RBISs into their teaching.At this point in the project we can not yet make claims about our broad research questions (i.e.Does the FOLC promote sustained change?), but the results reported here are early indications of success.
The majority of interviewed participants reported joining the FOLC due to a need for a community of support.Moreover, half of the participants joined the FOLC to get help with implementation of teaching reforms, which indicates that they expected to run into challenges and that they wanted the FOLC community to help them overcome those challenges.This motivation aligns with the community-based change model that underlies the FOLC in that the participants recognized that they would be more successful at achieving their reform goals with the backing of a supportive community.Additionally, the fact that nine participants reported not having such a community at their home institutions highlights the need for the FOLC to fill that gap.
But did the FOLC community achieve improved outcomes for the participants?When discussing impacts, nearly all interviewed participants reported that the FOLC experience was valuable, all full-year cohort participants reported changing their teaching as a result of the FOLC, and the majority of participants reported increased knowledge about research-based teaching.These outcomes align with our desire to improve the teaching of the FOLC participants.
It is encouraging that a large fraction of the participants spontaneously reported that the FOLC increased their confidence to try new things.Instead of giving up on new techniques when challenges arose, participants described turning to the group for support and advice and then continuing on with the change because the FOLC gave them the confidence to do so.Other participants reported that the FOLC made them more adventurous, such that they tried teaching strategies farther outside their comfort-zone.This is not an outcome that one would expect from a traditional D&D-based change effort, and illustrates the power of community-based interventions on the development of individuals.
The fact that many participants described engaging in reflective behavior is a promising sign for the long-term effect of the FOLC.Participants who completed SoTL projects got practice in evaluating whether a teaching change was working, and all participants talked to their cohort about what was going well and what was challenging in their classrooms.This increased awareness of one's teaching should help participants in overcoming implementation challenges, identifying new strategies to try that align with their goals, and modifying strategies to align with their context.These critical skills are not developed in most D&D-based change efforts.

VI. CONCLUSION
Participants' self-reports indicate that the FOLCs are a promising mechanism for sustained and effective reform of faculty teaching practices because of the supportive community that the FOLC provides to its participants.Because of this community, FOLC participants are trying new teaching techniques, learning how to navigate implementation difficulties, and developing reflective teaching skills.We look forward to offering more FOLCs in the future and continuing to research them to help guide best practices in the design and implementation of similar programs.

TABLE I .
Total number of participants who completed each FOLC by cohort and the distribution of gender (female vs. male) and institution type (offer graduate degree in physics/astronomy vs. offer only undergraduate degree) among those participants.

TABLE II .
Self-reported impacts of the FOLC on the 23 interviewed participants who completed the FOLC (percentages rounded to the nearest 5%).encouraging them to reflect on their teaching more than they would have otherwise.We define reflection as stepping back and considering what is working in the classroom and what is not, and then thinking about what adjustments could be made.Reflection also includes thinking about how one will assess if what one is doing is working.In talking about how the FOLC has met his expectations, one participant says he has been "learning new ways of doing things that are different at different institutions, because they have such different cultures, and then trying to see how that could work in the culture here or in my style, just kind of a reflection of what can I do better in, what things am I doing well, and seeing how other people have answered those questions and what are the questions that I should be asking myself."