Mobile Making : A program to broaden participation in Making

Making is a design-based, participant-driven endeavor based on a philosophy of “learning by doing.” Formal and informal educators at science museums, libraries, and universities see Making as a way to engage youth in authentic experiences involving science and technology. However, there is a lack of ethnic, gender, and socioeconomic diversity among Makers. Drawing on research-based principles for engaging underserved com-munities, we have designed and implemented Mobile Making, a program that operates in afterschool programs at four nearby middle schools. Teams of highly qualified and ethnically diverse undergraduate science and math majors lead weekly sessions that introduce participants to a variety of authentic Making activities. Program objectives target increases in the participants’ interest and self-efficacy related to Making and STEM, as well as their perception of the relevance of STEM/Making in everyday life. Observations, focus groups, and surveys have documented positive impacts on the participants and the undergraduate facilitators.


I. INTRODUCTION
Making is a design-based, participant-driven endeavor based on a philosophy of "learning by doing."Propelled by their own curiosity, Makers look for new ways to solve problems and then share their ideas with others.Making is inherently interdisciplinary; most activities cross boundaries between subject areas.Students acquire a deeper understanding of physics, computer science, engineering, math, and materials science through their direct experiences with both high and low tech materials.This open-ended, project-based approach to learning has caught the attention of informal educators, who see Making as a means for inspiring the next generation of innovators.As university faculty, we value Making as a way to promote interest in science and technology, provide relevant mastery experiences, and develop an increased sense of ability to contribute in these fields.Yet, recent reports on the demographics of the Maker community -which describe the typical Maker as a white male with a graduate degree and an income of just over $100, 000 per year -point to inequalities in access to resources and raise questions about the relevance of Making outside of its upper-middle class roots [14].Furthermore, while Maker Spaces help engage and support participation in Making, access may be limited because of proximity or cost [1].
The lack of access and diversity within Making has resulted in a desire to establish Making programs in underserved urban and rural communities.This shift in focus presents logistical barriers; more fundamentally, there is limited understanding of how the culture and practices of Making resonate with members of underserved communities.As a result, there is no guarantee that existing Making programs are relevant and sustainable outside of their original context and participant groups.In particular, there is a need to purposefully examine how Making activities can be implemented in underserved communities and to identify the strategies needed to sustainably support Making at scale in different settings.
California State University San Marcos (CSUSM) is a regional university located in north San Diego County, an eth-nically diverse area with many potential first generation college students.The CSUSM STEM Center operates outreach programs designed to provide local youth with STEM experiences and connections to near-peer mentors who are positive role models, and, more broadly, foster university-community linkages.As part of these efforts, we have developed a Mobile Making program with the goal of broadening participation by taking Making to underserved communities in existing afterschool programs.The program design is based on principles for engaging underserved audiences in Making.Program evaluation conducted over the two-year pilot effort indicates the effective instantiation of the design principles and positive impacts on the participants.

A. Design principles
Sociocultural and self-efficacy theories, which form our framework for understanding Making, guided the development and evaluation of the Mobile Making program.Sociocultural theory describes learning as social process within a cultural context involving development of tools and skills [12,22].Self-efficacy is a belief in one's capability to succeed in a specific context, and is developed through mastery experiences, vicarious learning, and verbal persuasion [5].
From this perspective, we identified six design principles for engaging underserved communities in Making.These served as the basis for designing the Mobile Making program.Access to resources is clearly important, and is our first design principle.However, addressing a lack of resources is not the entire answer.Our additional research-based design principles identify the importance of: 2) ethnically diverse near-peer leaders, 3) authentic activities, 4) ongoing input/guidance from participants, 5) legitimacy within the community, and 6) university-afterschool site partnerships.

Design Principles
Research -Underserved Youth 1. Access to resources [8,21] Schools in poorer urban areas are more likely to have inadequate curricula, under-qualified teachers, and fewer instructional materials 2. Ethnically diverse nearpeer leaders [8,15,20,23] Peer leaders help underrepresented students and girls negotiate feelings of isolation and stereotypes Mentors provide adolescents with a greater sense of value and help them envision a future in STEM Peer models are especially useful in situations in which young people have little experience with which to form a judgment of their competence.3. Authentic activities [4,16,17] Activities that engage students in applying science and engineering practices to real-world issues, can be especially impactful for underrepresented students and girls.4. Ongoing input from participants [10,13,18] Important for students to assume ownership of activities or projects Students experience an increased sense of belonging when they have a voice in a project design 5. Legitimacy within community [7,9,11,12] Parental involvement influences students' interest in science and connectedness to the academic environment Community engagement helps students see the relevancy of science in their everyday lives Adolescents' interests, attitudes, and motivation toward science, depend on community beliefs, acceptable identities, and other consequences related to life outside school 6.University-site partnerships [3] Collaborative, two-way partnership helps support and sustain engagement Site personnel (K12 teachers, administrators, and afterschool coordinators) are essential to effective programs

B. Program description
Guided by the design principles, we developed a Mobile Making program.For the past two years, we have operated the program in afterschool programs at four local middle schools in the Vista Unified School District (VUSD), located near CSUSM.These schools are located in ethnically and socioeconomically diverse areas; between 45% and 90% of students are Hispanic/Latino and/or qualify for free or reduced lunch.By operating in existing afterschool programs, we take Making to the communities we seek to engage, thereby addressing the first design principle, access to resources.
The second design principle recognizes the importance of near-peer mentors.At each afterschool site, teams of CSUSM undergraduate students, termed STEM Ambassadors, lead weekly, two-hour sessions that introduce participants to a variety of authentic Making activities.The Ambassadors bring materials, equipment, and tools directly to the school sites.These ethnically diverse leaders satisfy a key program design element because having a mentor who "looks like me" can be transformative for some minority students [6].As a regional university, CSUSM attracts a high number of students from the local area, meaning the undergraduate facilitators are drawn from the same demographic pool as the participants themselves.More than 50% of Ambassadors self-identify as members of underrepresented groups.Although the undergraduates are an essential "workforce" for the program, they also benefit from the experience by increasing their technical knowledge and leadership skills.
The third and fourth design principles underscore the importance of authentic activities and ongoing input and guidance from participants.Making activities tend to be, by their nature, authentic tasks with opportunity for participant input and choice.In the Mobile Making program, we gather feedback from the participants and facilitators and modify the ac-tivities to ensure they are engaging and accessible.We developed several different Making activities that are compatible with the format and time constraints in after-school clubs [2].Through their engagement in projects related to robotics, electronic textiles, solar powered vehicles, toy hacking, and game development, students learned about circuits, mechanics, energy, and basic coding.For example, in one popular activity, participants created original board games using LEDs, buzzers, and motors, which required investigating and troubleshooting circuits.Upon completion of each project or activity, student feedback was collected via a short form.
The fifth and sixth design principles relate to the program's relation to the local setting, including the community and school site.The Mobile Making program operates out of the CSUSM STEM Center, which since 2010 has developed relationships with teachers and administrators in VUSD.Building on these relationships allowed us to operate in existing afterschool programs, which has conferred some legitimacy on the program.To foster community engagement and support, Ambassadors presented Family Maker activities for parents and families during the schools' parent nights.Students' projects were also featured at the school district's annual STEM event as well as local Maker Faires and science and engineering festivals.These efforts were designed to promote the value of Making and science within the communities.

III. RESEARCH QUESTIONS, DESIGN, & METHODS
In seeking to understand the Mobile Making program as a mechanism for broadening participation in Making, we pose research questions related to impacts on the middle school youth as well as undergraduate facilitators.1) How does the middle school students' participation impact their interest, self-efficacy, and perceptions of the relevance of science and technology?2) To what extent do the undergraduate facilitators develop technical or other skills, and find their involvement meaningful?Given the nature of the program, study design followed an ecological approach, collecting impact data while also providing formative feedback to guide program implementation.Participant impacts were assessed by a mixed methods approach involving observations, focus groups, interviews, repeated measures surveys, and attendance numbers.Students' sustained interest and participation in Making (and other STEM-related activities) was assessed using attendance, weekly feedback forms, observations, and focus groups.A time series-based observation protocol was used to document students' making efforts.Each observed student's activity was observed and recorded every 30 seconds.Observations recorded, for example, engagement, interaction with peers and STEM ambassadors, and physical location within the classroom.
Changes in students' self-efficacy and perceptions were measured using a pre/post inventory.Items related to selfefficacy followed the recommendations outlined in [5], which suggests that scales of perceived self-efficacy must be "tailored the particular domain of functioning that is the object of interest or else risk ambiguity about exactly what is being measured."Students' perceptions of the relevance of science were measured on the pre/post survey using a modified version of the Changes in Attitudes about the Relevancy of Science (CARS) instrument [19].Like the self-efficacy scale, the relevance instrument included Making-specific items and more general STEM-related items.A post-only instrument was used to better understand the relative impact and importance of the design principles.This Design Principle Inventory (DPI) was an evaluator-designed, project-specific instrument that was produced and tested during the pilot phase of the program.In a Likert format, students were asked to respond to statements related to the instantiation of the design principles.For instance, items related to design principle 2, the importance of ethnically diverse near-peer leaders, included: the program "allowed me to work with college students who I could relate to," and "helped me see career possibilities for people like me in science or technology." Pre/post surveys and focus groups were the primary means of evaluating the program's impacts on STEM Ambassadors.Surveys, designed by the program evaluator, utilized selfreport data to measure constructs related to leadership, content knowledge and self-efficacy with regard to mentoring middle school students and teaching STEM content.The evaluator also led focus groups with all Ambassadors at the end of each semester.These debriefs allowed for additional insight into trends revealed in the pre/post surveys.

IV. OUTCOMES
In the program's first two years, over 200 middle school students and 46 undergraduates participated.Our research questions focus on the impacts on the middle school stu- In-person observations conducted by the program evaluator suggest that students were very interested in activities associated with Making.During a series of 20 sessions, observers found that students were on task and engaged in their projects, on average, 89% of each observed time period.Feedback from STEM Ambassador focus groups provides further evidence of students' interest in Making.Ambassadors from each afterschool program site described how students were adamant about taking their projects home to share with their parents (consistent with Design Principle 4).At times, when this wasn't possible due to the cost of the project parts, students expressed their disappointment.On a post survey, when asked to rate the importance of "Having opportunities for my parents to see the maker projects I've completed," 78.3% of respondents (n = 49) indicated it was either "very important," or "critically important." Data from pre/post surveys and feedback forms provide evidence of changes in students' self-efficacy and perceptions of the relevancy of science.Using a five-point Likert scale, students were asked to rate their agreement with 22 statements designed to specifically target self-efficacy and relevancy in Making-style programs.While students' post responses were higher for nearly all of the 22 items, the gains were statistically significant (paired t-test with p < 0.05) for five statements, shown in Figure 1.

B. Undergraduate participants
Evaluation results suggest that undergraduate STEM Ambassadors gained (1) content knowledge, (2) leadership and communication skills, and (3) a sense of personal accomplishment and positive contribution to others.Ambassadors' content knowledge was not directly assessed (this remains a topic for future work).However, observations and participant outcomes indicate that the Ambassadors were able to successfully lead the Making activities, which involved circuits and other physics and technology topics.Many of the undergraduates were biology and biochemistry majors whose experience with circuits was limited to their introductory physics courses.Their training and work with other Ambassadors prepared them to effectively facilitate the Making activities.It is thus likely that this improved their understanding of concepts in physics and technology.
Ambassadors also honed their leadership and communication skills as they gained more experience facilitating the after-school activities.In focus group sessions, they each described individual experiences working with student participants.The afterschool setting often proved challenging, with regard to focusing the attention of some students and working to engage and sustain their interest to the point of success.As a result, STEM Ambassadors demonstrated significant abilities to adapt to varied and, at times, chaotic situations.Along the way, they developed abilities and strategies for working with middle school students, and generally engaging others in the Making effort.
STEM Ambassadors completed pre-post Likert surveys on their excitement, level of preparation, ability to contribute, and enjoyment related to the program.Unsurprisingly, given that they applied and were selected for the role, Ambassadors responded positively about these areas.More significantly, their responses were more positive or constant on post assessments, after having worked with middle school students for an entire semester.These mentors maintained a high level of enthusiasm, confidence, and satisfaction with their rolesan accomplishment after having faced the realities of working with middle school students in afterschool settings.

V. CONCLUSIONS
Making is an exciting, authentic way to engage learners in science and technology.However, there is a lack of ethnic, gender, and socioeconomic diversity among Makers.The Mobile Making program is designed on research-based principles for engaging underserved communities.Undergraduate facilitators, math and science majors who serve as nearpeer mentors, are a key element in the program.In the past two years, over 200 youth engaged in Making through afterschool programs at four nearby middle schools with diverse student populations.We have documented positive impacts on student participants' interest levels, perceived relevancy, and self-efficacy specific to Making.Undergraduates benefit through increased content knowledge and enhanced leadership and communication skills.
Table I summarizes these principles.

TABLE I .
Design Principles for Mobile Making program.