Evaluation of a Tertiary Studio Physics Implementation in Abu Dhabi

As part of an international collaboration on educational dissemination, the Colorado School of Mines Physics Department is involved in the evaluation of a tertiary implementation of the Studio Physics environment at the Petroleum Institute in Abu Dhabi, UAE. This paper will describe the preliminary results on student performance based on traditional means of summative assessment, the Force Concept Inventory, a Studio Learning Environment Survey, and semi-structured interview data with eight physics faculty and staff and eighteen current and former students regarding their opinions on introductory physics and the Studio environment and methodology. The discussion will also involve comparisons of different classroom cultures, context idiosyncrasies and their potential impact on the reform effort.


BACKGROUND
The Studio Physics model [1][2][3][4] and related models [5,6] take a comprehensive approach to course reform, with a complete restructuring of the physical environment in addition to changes in pedagogy and curriculum. These approaches aim to teach physics with most or all of the contact time taking place in a room designed to facilitate interactive engagement: the Studio. While the physical layout of the Studio varies between adoptions, all are designed to encourage and maximize intra-and inter-group cooperation through arrangement and proximity of groups. Furthermore, curricular materials are designed to explicitly facilitate student interactivity. Although each of the above mentioned innovations has its own pedagogy and curriculum, the physical environment itself can be successfully adopted in a variety of contexts to match the level and goal of the curricula. [7,8] The Colorado School of Mines (CSM) was an early developer of a hybrid-Studio environment in its introductory physics courses. [4] Continued study of our Studio courses led to the development and dissemination of a model for adapting this learning environment to a traditional course in a manageable, reproducible manner. [9] As part of the dissemination of this work, Kuo and Kohl recently collaborated with the Physics Department at the Petroleum Institute (PI) in Abu Dhabi. Barriers to transference and adaptation of reform efforts in STEM disciplines have recently been well-documented. [10,11] This paper will attempt to identify and describe some of the cultural, contextual, and constraint influences that exist in this Middle-Eastern tertiary implementation of Studio Physics.
The PI is an English-medium higher education institution in the city of Abu Dhabi, United Arab Emirates. Established under a collaborative effort between CSM and the Abu Dhabi National Oil Company (ADNOC) in 2001, it is a premier institution in the region with the explicit goal of preparing the next generation of Emirati engineers. Students at the PI receive full scholarships for maintaining satisfactory progress towards degree. Graduates are then employed by one of the companies of the ADNOC group. The PI has separate male and female campuses with 3 academic levels. Prior to formal enrollment into the university, students start in The Advanced University Placement Program, a one-year college preparation course. Successful students become Freshman and shift to The Arts and Sciences Program (A&S) for a 2-year sequence covering all core courses. Finally, students transfer to The College of Engineering to take a 2year course sequence in one of the five engineering departments. The degree programs have international accreditation through the Accreditation Board for Engineering and Technology (ABET).
The Physics Department is in A&S, and is responsible for delivering the calculus-based introductory Mechanics and Electromagnetism courses for all PI students. The Department had five full-time faculty and 4 full-time educational support staff at the time of this study. The curriculum had been fairly traditional (teacher-centered with separate theory lectures and laboratory sections), and recent high Drop-Fail-Withdrawal (DFW) rates led to the consideration of possible educational reform.
Prompted by a visit to CSM during the summer of 2011, Bradley, Beks, and Agyeman began developing an adaptation of the studio learning environment and curriculum for the introductory Mechanics course, and the initial pilot implementation took place with a single section on the female campus. Perceptions of the potential of this reform effort led, in the summer of 2012, to the construction of two dedicated studio classrooms, funded by Takreer, the refining company in the ADNOC group of companies. These rooms can accommodate 36 students each. Kuo was invited as a visiting professor during the fall of 2012 to further collaborate and develop the reform effort.
The PI has been fluctuating in its enrollment recently, and the number of students taking physics courses each semester varies significantly. For example, for the introductory Mechanics course, the Fall 2012 semester had around 50 students (two sections on the male campus and one section on the female campus), while the Spring 2013 semester had 120 students (three male and three female). All introductory Mechanics courses were taught in the hybrid lecture/studio format starting Fall 2012. Each section had anywhere between 15 and 34 students and was taught by a faculty member and a support staff person.
The students' prior educational experiences, and thus expectations of the learning environment, are of teacher-centered content delivery, explicit provision of examples, and assessment of knowledge and understanding that rests primarily on what may be considered as direct regurgitation. Prior academic success under this format makes the student-centered, individual ownership and responsibility learning model that underlies the studio philosophy an extremely unfamiliar classroom culture shift for the students. There were initial concerns and hesitations due to this perceived barrier.

COURSE AND METHODS
The course, based on context history and schedule constraints, involves 5 meetings each week: 50minute lectures on Sundays and Thursdays, with studio sessions on Mondays, Tuesdays, and Wednesdays of 75, 50, and 75 minutes respectively. This schedule maintains the normal contact time for lab-based, 4-credit courses at the PI.
The lectures, while incorporating Peer Instructionstyle [12] conceptual questions to facilitate student engagement, were otherwise fairly traditional. The studio activities incorporated several different formats, including experiments, problem solving, and exploration and analysis of physical phenomena using the PhET [13] simulations. The studio pedagogy utilizes a locally developed and adapted version of progressive removal of scaffolding. [14] In order to evaluate the reform effort, DFW rates and final exam performances are analyzed. The Force Concept Inventory (FCI) was also used to provide for more standardized comparisons. A Studio Learning Environment Survey (SLES) was developed and administered by Bradley during Spring 2013 to gauge student perceptions of the course. The survey asks students to provide free-response answers to six questions about aspects of the Studio Physics learning environment such as working in groups and the level of teacher support. Students are first asked to answer based on how things actually are in the classroom. Then, students are asked to indicate their preferences related to each item, assuming things could be adjusted as they like. Also  Since the dominant assessment in the course is the final exam, it was viewed as a plausible place to seek evidence of possible learning gains attributable to course reforms. While the final exam questions change each semester, the format and topical 214 coverage of the exams remains constant. The course instructors write the exams collaboratively and attempt to maintain a consistent exam difficulty for most topics. The final exam consists of a few freeresponse questions, ranging in difficulty, for each of the eight major topics. While there are variations, performance trends are similar to that shown in Figure 2, with students from both traditional and studio approaches performing better on early questions and a bit worse on the later questions. The average score is obtained from all of the sections taught in the traditional manner between Spring 2009 and Fall 2012 (n = 514). This is compared with averaged scores from Studio Physics sections between Fall 2011 and Spring 2012 (n = 107). The standard deviation of item scores ranges from 21% to 31%, with a typical value 26%. For comparison of the average scores of the two groups, a 95% confidence interval of approximately 2 √ ⁄ ( 5% or 2.5% for the studio and traditional groups, respectively) is shown. An initial conservative interpretation of the data is that the change in course format did not harm student performance. For six of the eight topics, the studio group's performance is essentially the same as the traditional group's, judging from the substantial overlap in the confidence intervals for those items. For two topics, Energy and Rotation, the studio group appears to have done slightly better than the traditional group. We conclude that so far, exam performance shows no significant improvement attributable to the course format change.
To further explore the issue of learning gains, the FCI was administered in both the traditional and studio groups. In Fall 2012, for students who took both pre-and post-instruction surveys, the preinstruction score average (standard deviation) was 27.2% (11.8%) for male studio students (n = 29) and 22.5% (11.5%) for female studio students (n = 9). The normalized gain average (standard deviation) was 21.5% (15.4%) and 11.6% (7.5%) for the male and female studio students, respectively. In past semesters with traditional instruction, average preinstruction scores of 20-25% and normalized gains of 15-20% were typical. While these results provide no evidence of improved conceptual learning, they do seem to hint at a gender difference that we plan to investigate in the future.
The SLES was administered in Spring 2013. A total of 120 students participated in the survey. On the question of Working in groups to learn physics, the responses were 73% positive, 9% neutral, and 18% negative. On the question of Level of teacher support in class, the responses were 65% positive, 15% neutral, and 20% negative. There were no significant differences in actual versus preferred perceptions on these items. Responses on the other four questions are consistent with these results.
The interviews with faculty and staff ranged from 20 minutes to over an hour in length. The data indicated diverse and varied perceptions on the purposes, philosophy, and effectiveness of the Studio environment and methodology. While the faculty and support staff members basically agree on what indicates student success as manifested in the various measures and assessments in the course, they differ in their opinions on how to facilitate student learning to achieve the desired outcomes.
The interviewees who placed high value in having students actively engaged with curricular material and taking ownership of their learning were strongly in favor of the interactive environment of the Studio, and expressed their agreement with the large fraction of student contact time being spent in the Studio Physics environment. Those who expressed concerns about the students' abilities to transition and adapt to the new learning style, based largely on their perception of the students' background, preferred to have more contact time to demonstrate and model problem solving analyses and processes for the students in traditional lecture formats.
The student interviews included seven female students and eleven males, and ranged from 25 minutes to over an hour in length. Of the women, 5/7 had strongly or slightly favorable views of Studio Physics at the PI. One had a neutral opinion, and one had a slightly unfavorable view of Studio. Of the men, three had participated in Studio in a previous semester and had a strongly favorable view of Studio. Eight were current Studio Physics students, and 7/8 of those exhibited a strongly unfavorable view of Studio, with 1/8 exhibiting a strongly favorable view (that one being from a different section than the other seven). We observed no clear correlations between favorable/unfavorable views and such factors as major, AUP participation, and high school system.
In general, satisfied students praised Studio for the emphasis on group work, the amount and depth of material covered and what they actually learned the relation to real-life applications, and the time management skills that they developed in response to the course demands. Unhappy students compared Physics to their Chemistry and Calculus classes, both taught in traditional mode. Their primary complaints involved workload, instructor preparation and attitude in class, perceived inconsistencies in grading, and the infrequency of lecture.

DISCUSSION AND CONCLUSION
The assessment data indicate either consistent or slightly improved student performance as a result of the course reform. Student responses to surveys and interviews were mostly positive, with no significant evidence of barriers to accepting the new pedagogy and culture of learning.
Interviews revealed that a number of faculty and staff had initial concerns and hesitations that inadequate student preparation would result in student failure and dissatisfaction under the reformed format. Students who expressed dissatisfaction with the new environment were taught by reluctant faculty and support staff. The students who provided positive interview responses were from sections where the faculty and staff wholeheartedly embraced the new pedagogy and continually articulated the purposes and benefits of the new approach to the students.
The reform effort faces a number of significant challenges including students' experience of a traditional classroom culture, a traditional educational context (other courses) at the PI, and minimal preparation in learning responsibility. Consequently, in their Studio Physics course at the PI, students encounter a seemingly massive shift to a unique and unfamiliar culture of learning. Despite this, initial results indicate that the largest barrier to this reform effort may be faculty and staff perceptions of student capabilities and needs, and thus their reluctance to facilitate such changes.
The data and results reported in this short paper are rather cursory. A future, full-length paper will include more complete and detailed analysis of both quantitative and qualitative data.