Team-based learning in physics courses

A study of the impact of team-based learning (TBL), an active learning teaching strategy developed by Larry Michaelsen of the University of Oklahoma, was conducted to determine the impact of TBL on students in introductory and upper-level physics courses. The study was designed to use institutional data to determine whether TBL increase retention of physics majors. A survey, the physics TBL survey (PTS), was developed and used to understand student experiences with TBL in the classroom. We found that TBL increased retention of physics majors overall and specifically in the critical freshmen and sophomore years. The results from the study show that students value TBL favorably for both introductory and upper-level physics courses over 8 different measures. Introductory students ranked the learning of physics, the effectiveness of the TBL format for physics courses, and the value of the team experience higher than the upper-level physics students. Finally, students indicate that TBL is a less intimidating classroom format, they appreciate instant feedback because it helps them to learn more quickly and keeps them engaged in class, they find peer explanations of new physics topics to be valuable, and they are find that the readiness assurance process keeps them motivated and accountable.


I. INTRODUCTION
Larry Michaelsen, a management professor from the University of Oklahoma and a strong proponent of Active Learning, formally developed Team-based learning (TBL) in the 1970s to continue using interactive methods with much larger classes. He implemented successful team building skills in his courses with five critical components that include: teams of 5-7 students that stay together for the entire semester, participation in a Readiness Assurance Process (RAP), in-class building activities to develop high performance teams, immediate feedback and assessment, and grade weighting that includes team component [1].
More recently, Swanson et al. [2] conducted a metaanalysis of 17 published studies of TBL and reported improvements in measures of content knowledge with TBL instruction for both undergraduate and graduate students in courses in medicine, biology, economics, neurology, and pharmacy.
Team-based learning is used effectively in medicine, economics, nursing, chemistry, and biology courses [3][4][5][6][7]. Much of the early literature on TBL in education reported positive outcomes in medicine and nursing. Despite the development of active learning in physics, there are only a few studies of TBL in physics. Toggerson et al. submitted a paper in January 2020 and reported a positive impact on student self-efficacy for identifying students in physics of life science course at the University of Massachusetts [8]. Parappilly and her colleagues apply TBL pedagogy in introductory physics courses for engineering students and physics courses for nonscience majors. Parappilly substitutes 4 traditional activities and replaces them with TBL activities in her courses [9] and reports positive learning outcomes.

A. Motivation for TBL in our Physics Courses
The authors have many years of experience with active learning strategies and were already implementing active learning strategies including group activities and Just in Time Teaching preparation techniques in their physics classrooms [10]. However, as in many physics departments throughout the United States, the goal is to increase retention of physics majors, increase student preparation before class, improve learning outcomes, and create a welcoming and engaged community of physics learners. A TBL workshop offered by the Center for Teaching and Learning at the college during winter break in 2017 seemed to be the next step in achieving these goals. The benefits of TBL aligned with the physics department's desire for improvement and the practice of TBL seemed like a reasonable next step.
The authors felt that TBL had the potential to positively impact both lower-and upper-level students. In TBL, faculty are encouraged to define the skills that students are expected to develop through the course and create team activities that develop these skills. For example, many students practice simplistic problem solving techniques. The goal is for students to spend some time with the problem, identify the physics model, generate solutions to recognize the relation between the variables, and then make sense of their answers. By guiding the teams through group activities that require more sophisticated reasoning and problem-solving, students understand the value of a more comprehensive problemsolving framework. With enough practice students internalize the process and can use it effectively.
The physics faculty were interested to see if TBL would help increase retention of freshmen and sophomore physics majors. High impact practices such as learning communities and collaborative assignments and projects are shown to increase persistence and retention for freshmen [11]. Existing literature is lacking for retention beyond the first year, specifically for physics majors. However, recently, high-impact practices were found to be successful generally for sophomore students [12].

B. How TBL Works
TBL guidelines include: 1. Strategically-formed teams of 5 -7 students that stay together for the entire semester, 2. Require before-class preparation through a RAP, 3. Team activities that provide opportunities for teams to develop into self-managed, high-performance teams, 4. Immediate feedback and peer assessment, and 5. Course grades that are weighted by team performance. Students are carefully assigned into permanent teams of 4 -7 students in such a way as to distribute resources equally among the teams. This may be done using the CATME Comprehensive Assessment of Team Member Effectiveness system developed at Purdue University. CATME is designed to create equitable teams and administer peer evaluations of team members [13].
The team-based learning sequence is then applied to each unit in the course. Each unit will follow an instructional activity sequence that includes a RAP, team applications of course concepts, and a final assessment. As part of the RAP, students are assigned outside work to prepare for the new unit. Students take an individual test and a team test with immediate feedback during the first class of a new unit. Most courses include 4 or 5 units with varying topics in each unit and the team activities vary greatly across the courses.
In most implementations of TBL, the individual and team tests for the RAP are short assessments that are mostly conceptual questions on the new content in the upcoming unit. Simple numerical problems may also be included. After the teams complete the team test, they can immediately check their individual and team answer choices against the correct answers for immediate feedback. The instructor then addresses any common areas of difficulty before moving on to the team activities.
The team activities are designed to require the participation of all members of the team. Well designed team activities are critical to the success of TBL. While the problems may start off simply, more complex real-world problems are asked during this phase so that discussions are rich and all viewpoints are needed to make progress in these activities. Teams become more effective at solving complex problems as they spend more time together.
Finally, individual and team summative assessments are given at the end of each unit to evaluate individual student and team understanding of the content. The team tests are more complex than the individual tests and require all members of the team to participate.

II. METHODOLOGY
A larger TBL study is ongoing by the authors at a small liberal arts college in the northeast. Parts of the larger study are included in this paper. Research questions include: 1. Is there a difference in retention of physics majors for students in active learning physics classrooms compared to TBL classrooms? 2. How do students rank TBL with respect to preparation, learning, accountability, performance, engagement, team experience, course format, and motivation? and 3. How do students describe their experiences in a TBL classroom with respect to the above list?

A. Aggregated Retention Data
Aggregated institutional data were provided by the college to compare physics majors retained in the major before and after TBL was implemented. The measure of retention is the number of physics majors who changed their major after taking a physics course. The four authors converted their courses from interactive engagement classes to TBL classes. Courses include introductory and upper-level physics courses. Of the courses considered for TBL, the calculus-based introductory sequence and upper-level physics courses were chosen as these were the courses taught by the 4 authors. A listing of the courses and when they were offered are provided in Table  I. All 4 credit courses include a lab. Students participate in labs in their teams except for General Physics I II. Retention was compared for these courses before and after TBL was implemented. It should be noted that there are additional students in the physics major who are not included in the dataset as they were not taught by the TBL authors. The numbers of students in each cohort are listed in the results section in Table  II. The data from all courses was combined for the analysis to provide power and sample size for the statistical analysis.

B. Physics TBL Survey (PTS)
The authors designed a mixed-methods study to understand how students viewed the effectiveness of TBL in both lower-level and upper-level physics courses. A locally selfdeveloped survey was administered that includes Lickertscale questions and free-response questions about TBL. The survey is called the Physics TBL Survey (PTS) and the validity, reliability, and fairness of the survey will be published separately.
The Lickert-scale and free response questions were developed on TBL topics including preparation, learning of physics, accountability, performance (grade), engagement, team value, motivation, and TBL format. The survey asked specific questions about preparation for and accountability to the team for graded group work included in the TBL process. The Lickert-scale responses ranges from 1 to 5 with 1 representing strongly disagree and 5 representing strongly agree. For the analysis, a score of 3 or above is considered a positive response to the question.
The free response questions asked students to describe their experience or opinion on the 8 TBL topics listed above. A qualitative phenomenology framework directs the thematic analysis of the answers to the free-response questions [14]. Using this phenomenology lens, the common ideas for each topic were coded to understand the essence of the student experience in a TBL physics course.
The Physics TBL Survey (PTS) and the coding of the questions is available on the college website [15].

C. Participants
Students in the lower-level courses include students majoring in physics, chemistry, biochemistry, math, and environmental science. In the upper-level physics courses, most students are physics or applied physics majors. It must be noted that the aggregated institutional data described in section IIA, only includes physics majors. The mixed-methods part of the study considers ranking and opinions of physics students and students that take physics to meet auxiliary courses requirements for their major. The total number of students that participated in the study is 260 with 153 students in the lower-level physics courses and 107 students in the upperlevel physics courses.

A. The Effect of TBL on the Retention of Physics Majors
The analysis of the retention of physics majors is included in Table I for courses before TBL was implemented and after. Due to power and sample size considerations, the data from all courses taught by the instructors before TBL was implemented were combined. The data from all courses taught by the instructors after TBL was implemented were also combined. The major of each student was recorded before and after taking the course in question. The measure of retention in physics is given as a percent of the number of students that continued as a physics major after taking the course. The values in the column labeled as "n" in the Table II are the number of physics majors in courses taught by the 4 authors. Before TBL was implemented, there were 105 physics majors in all courses and 85 were retained (81%) at the end of the courses. After TBL was implemented, there were 110 physics majors in the TBL courses, and 100 were retained (90%).
Freshmen and sophomore students were combined for the analysis, and retention increased from 68% to 80%. Retention of junior and senior majors was already near 100%, and TBL did not impact retention for these classes. This is not surprising given that very few science students will change their major after sophomore year.
To determine whether these increases are statistically significant, a 2-sample proportion test was conducted using a Z statistic test. The before and after group retention given as a percent was compared as shown in Table II. First, the difference between retention percentages of all students before and all students after TBL was determined to be statistically significant indicated by a p-value of 0.002. The same analysis was performed for the subgroup including the freshmen and sophomores. The difference between the retention rates from 68% to 80% was also statistically significant indicated by a p-value of 0.026. This indicates that TBL courses increase retention of physics majors in the critical freshmen and sophomore years.

B. Analysis of PTS Lickert-scale questions
To further understand the impact of using the TBL approach in physics, an analysis was conducted of the Lickertscale and free response questions. For the analysis,a score of 3 or above is considered a positive response to the question. Lickert-scale questions were grouped according to topics and the means of lower-level (LL) and upper-level (UL) physics courses are given in Table III. Overall, students had a positive experience with team-based learning in both lower-level and upper-level physics classes. The means for all topics are between 3.8 and 4.2. The means were slightly lower for students in upper-level courses than students in lower-level courses. However, according to a T-test, these differences were only statistically significant for the topics: learning of physics, the value of the team, and the effectiveness of TBL as a format for physics courses. For the remaining topics, the responses from the two groups were consistent with each other, and we could not reject the null hypothesis that they were equal. For those topics where the means were lower for students in upper-level classes, this could be attributed to the fact that many of the upper-level students had not taken a TBL introductory course and their expectation for a different teaching delivery method may have impacted their responses. Note: * indicates the difference between the means for students in lower-level and upper-level classes are statistically significant.

C. Analysis of PTS Free Response Questions
In the PTS, 4 of the 7 questions from the survey were considered for this study to illustrate student common experiences with TBL. These questions include: In the answers to the free response questions, common themes were echoed by the students. As exemplified by this quote, many upper-level students reported that TBL provided a less intimidating classroom format: "I think TBL in upperdivision courses makes the topic less intimidating. Knowing you can openly consult with your group and use them to help you through the material makes it a lot less daunting than if you were facing it on your own." Presumably they are comparing TBL with more traditional lecture formats.
Students appreciate the instant feedback because it helps them learn more quickly and keeps them engaged. Students appreciated the instant feedback by seeing the correct answers at the completion of the RAP process. One student noted, "The instant feedback is very useful because sometimes when I submit something, I will just be happy that it's done and I can forget about it. With instant feedback it is super useful because I hear about what I did wrong instantly. This makes me more attentive and more available to learn from my mistakes." Another student said that the instant feedback from teammates during team activities, "helps to understand material quickly, makes the class more interesting." On the topic of peer learning, many students shared that they benefited from explanations from their peers when they were learning something new and they appreciated the sense of community TBL provides. For example, a student commented that, "It allows for an explanation of topics by a student with a view similar to yours. It allows me to realize if I know something or not. It builds a sense of community, and I'm more likely to work with them outside of class when studying and I can trust them to explain something if I'm confused." Another student commented that, "I feel I take away alot more from the peer discussions than from the readings themselves." Preparation is part of the TBL process and students expressed that the process motivated them to prepare but they also were motivated to help their team, "I liked how everyone felt motivated to learn a topic since they didn't want to let the other people in their group down."

D. Negative Student Responses to TBL
There is typically one student per class that doesn't like TBL. They usually object to having to spend time teaching their team mates or complain that a team mate isn't prepared. For example, one student commented, "I do not like TBL for any physics classes because I do not like having to stop my learning to teach others things they should already know or have studied on their own time." Another student commented, "For upper level classes it provides an easy way out for students that are still struggling since they can just copy off of their peers." Another student commented, "I'd rather have solely professor taught classes." Finally, a student commented that "I feel I was placed with people who didn't match my level of care or understanding (either they understood more or less) and had difficult coming to my level, whether that meant "dumbing down" or trying to understand it more." Finally, a student commented on a dislike of the TBL grading approach, "I enjoy learning in a team, but I do not like being assessed as a team and having my work bring up the grades of other people." Many of the negative responses to the free response questions about TBL came from high achieving students.

IV. CONCLUSIONS AND FUTURE WORK
A study was conducted on the impact of TBL on the retention of physics majors and their opinions about TBL conducted through a local self-developed survey, PTS. Summarizing the results: 1. Institutional data indicate that TBL increases retention of physics majors overall, and increases retention during the critical freshmen and sophomore years.
2. Lickert-scale PTS data indicate that students value TBL favorably.
3. Upper-level students rate learning of physics, the value of the team, and the effectiveness of TBL as a format for physics course lower than lower-level students.
4. Free response theme analysis indicate that students find TBL is a less intimidating classroom format, appreciate instant feedback because it helps them to learn more quickly and keeps them engaged in class, find peer explanations of new physics topics to be valuable, and appreciate the motivation and accountability that the RAP process provides.
This study of TBL has raised many questions. Some questions relate to student learning outcomes and others relate to how students experience TBL. Future studies are planned to examine quantitative learning outcomes including conceptual assessments, grades, and other learning outcomes. In addition, an in-depth qualitative phenomenological study including interviews to obtain a richer understanding of the essence of teambased learning for the students will be designed and conducted.