What Do Students Want? Small Group Instructional Diagnoses of STEM Faculty

Small Group Instructional Diagnoses (SGIDs) are informal, mid-semester evaluations of courses, which the instructors request voluntarily. The facilitator of the SGID comes into a class, the instructor leaves, and the facilitator spends about 30 minutes with the students. The first part of the SGID is done in small groups of students. The facilitator asks students to consider two questions, "What are the strengths of this course?" and "What suggestions do you have to improve the course?" After students work in groups for 10 minutes to compose lists of strengths and suggestions, the facilitator calls the class back together as a whole, records the most important strengths and suggestions, and has the class vote on them. These results are then shared privately with the instructor of the course. We collected and analyzed student comments from 45 courses taught by 27 STEM instructors. We found that students value clear lecture and hands-on learning, fair and frequent feedback, flexible and caring instructors, organized classes and resources, and clear alignment between instruction and evaluation. Looking in more detail, we found that students perceived small classes to be more organized than large ones. Students in introductory classes provided the least amount of feedback. Students in introductory classes placed the most value on instructor characteristics such as support and caring about their success, while graduate students placed the least value on instructor characteristics. Finally, female STEM instructors received a disproportionately high number of comments.


INTRODUCTION
Joseph Clark, of the Biology Learning Resource Center at the University of Washington, developed Small Group Instructional Diagnoses (SGIDs) 40 years ago [1] in response to concerns about traditional endof-course evaluations. He had noticed that students did not take those evaluations seriously, and instructors frequently did not use the information obtained from them [2].
SGIDs are informal, mid-semester evaluations of courses. A facilitator spends roughly 30 minutes with the students without the instructor present. The first part of the SGID is done in small groups of students. The facilitator of the SGID asks students to consider two questions: "What are the strengths of this course?" and "What suggestions do you have to improve the course?" After students work in their groups for about 10 minutes to compose lists of strengths and suggestions, the facilitator calls the class back together as a whole, records the most important strengths and suggestions, and has students (individually) indicate their agreement or disagreement with the statements. These results are then shared privately with the instructor of the course, who can use them to adjust instruction as desired [3].
The literature reveals that students are enthusiastic about SGIDs [2], preferring them to end-of-course evaluations [4,5], and their motivation increases after they are performed [2]. Students also think more highly of professors after they conduct these midsemester evaluations [4].
Instructors find SGIDs useful for both their current and future classes [2], considering them more credible than end-of-term evaluations [6]. In many cases instructors are able to make changes immediately, in the same term, though some just use the opportunity to clarify their current policies with their students [1].
The three of us have conducted SGIDs at Miami University for many years. They are done under the auspices of the offices of Advanced Learning Technologies and the Center for the Enhancement of Learning, Teaching, and University Assessment, both units reporting to the provost in support of faculty development at the institution.
All instructors may request SGIDs at any point in their career; most do so before tenure and promotion as formative assessment of their teaching. We have found the students' comments, both their views of the strengths of their courses and their suggestions for improvement, to be quite informative. In this article we lay out our analysis of student comments on SGIDs for instructors of science, mathematics, engineering, and technology (STEM) course with the belief that they will be interesting to other instructors as well as provide implications for instruction.

METHODS
We asked 256 instructors who had SGIDs done in their classes since 2005 if we could use their reports as data. Of these 256 instructors, 140 (55%) gave us permission to use their reports. Of those, 27 were STEM instructors, representing 45 classes and 1086 students. In these reports, the students had offered 313 strengths of their classes and 297 suggestions for improvements.
All of these strengths and suggestions were entered into an Excel file and then were coded. Each of us coded at least half of the data, using inductive categories. In contrast to deductive categories [7], which are based on prior research, inductive categories are grounded in the data itself [8]. After coming up with categories for the strengths and suggestions on our own, we met, discussed the similarities and differences in our categorizations, resolved differences, and came up with common names for our categories. We ended up with six categories of strengths and six categories of suggestions.
We then examined the data by the size and level of the classes and by the gender identity of the instructors. These results are presented next.

RESULTS
The strengths and suggestions that the students brought up during their SGIDs were each divided into six categories. These categories and the percentages of the total comments they represented are shown in Tables 1 and 2, below. Many of these strengths and suggestions were coded into similar categories. The two categories that contained the majority of the responses were "teaching style / method" and "evaluation." Students offered both strengths and suggestions about teaching style and method, listing as strengths such things as "lectures well-planned and interesting" and "hands-on learning." They also had suggestions about how to improve teaching style and method, such as "Move slower during lecture to allow us to take notes effectively" and "Allow more time for us to work on in-class problems." Also important to students were concerns about evaluation. Students listed as strengths such things as "Instructor provides ample feedback on assignments" and "Daily quizzes help/force students to keep up with material." They also offered suggestions, such as "Give more weight to individual work to provide incentive for individual learning" and "Keep feedback constructive and positive." Next most frequent were responses about the strengths and weaknesses of class materials. These applied to materials given during class as well as outside of class time, as the students identified as strengths items ranging from "lots of examples of problems given" to "Instructor posts lecture material before class." One of these suggestions, "Provide the complete set of slides before exams," is almost the converse of the strength students noted, and another common suggestion was "Choose a more understandable textbook." Nearly as many responses pertained to instructor support and course policies. Students listed as strengths that their instructors were "approachable and always willing to meet" and "flexible and willing to adjust instruction." They had suggestions about course policies, such as "Allow use of an equation sheet for exams" and "Please be more patient with our questions, especially during office hours." Students offered both examples of strengths and suggestions pertaining to the organization of the course and of the instructor. Strengths included "Instructor provides clear outline for each class" and "website up to date and easy to follow." Suggestions included a near converse of that latter strength: "Post all of our grades online" and "Wait for next class to begin new examples; don't carry them over. Spend time on review instead." There was one category each of strengths and suggestions that did not overlap. There were several students who listed characteristics of their instructors as strengths, like "cares about our learning" and "Instructor is highly knowledgeable and highly motivated." Students did not make suggestions about improving the characteristics of their instructors.
Students did make suggestions about improving instructional alignment. Some were sweeping statements, like "More alignment needed between homework, problems, and exams," and others were finer points, like "Study guide did not reflect what was tested."

Class Size
The 45 classes in this sample were broken down into small classes (fewer than 15 students), medium classes (15-25 students), and large classes (more than 25 students). The percentage of strengths and suggestions given by the students in those classes is almost exactly proportional to the percentage of classes in each size range, as can be seen in Table 3. Looking at the categories of strengths and suggestions more closely, it becomes apparent that, although the overall percentages are quite proportional, the strengths and suggestions about organization suggest that students perceive their small classes to be more organized than their large ones, as can be seen in Table  4. Although 32% of the classes were large (>25 students), students from those classes offered only 6% of the strengths about organization. Students in small classes (<15 students), representing 17% of the sample, gave only 8% of the suggestions about organization.

Level of Class
The classes in the sample were also divided according to their levels: introductory, 100-level courses; upper-level undergraduate courses; and graduate courses. Again, the percentages of the total numbers of strengths and suggestions given by the students in those classes almost exactly aligns with the percentages of classes in the sample, as seen in Table  5. It appears that instructor characteristics are disproportionately unimportant to students in graduate classes, while instructor support and course policies are disproportionately important to students in introductory classes, as can be seen in Table 6. Although 38% of the classes were graduate level, only 14% of the comments about strengths relating to instructor characteristics came from graduate students. Meanwhile, though only 22% of the classes were introductory level, students in those classes gave 38% of the suggestions about instructor support and course policies.

Gender of Instructor
There were more male instructors than female instructors in this sample of STEM classes; only 30% of the instructors were female. However, students in classes with female instructors provided 40% of both the strengths of the course and the suggestions for improvement, as seen in Table 7. Female instructors received more than their share of comments about both strengths and suggestions in five out of the six categories for each. The exceptions are shown in Table 8.

DISCUSSION
Previous research has explored what is important to students in terms of effective teaching [9,10]. It also has shown that students remember ineffective instruction and instructors-in some cases, even years later [11]. Many of these previous themes resonated in the data we present here: Students value clear lecture and hands-on learning, fair and frequent feedback, flexible and caring instructors, organized classes and resources, and clear alignment between instruction and evaluation. This study adds to the previous literature by looking specifically at how student SGID feedback varies according to the variables of class size and level and instructor gender among STEM courses. Regarding class size, students perceived small classes to be more organized than large ones. Instructors of larger classes may understand this impression, in that it is more difficult to maintain everyone's attention in larger classrooms with more students. Just being seen and heard, let alone offering personal attention and engaging student-student or student-instructor interaction, can be challenging. Class level also revealed interesting differences. Introductory-level students offered the least amount of feedback. This could relate to their stage of development compared to their generally older counterparts in upper-level classes. These students also placed the most value on instructor characteristics such as support and caring about their success. Graduate students, at the opposite end of the educational process, placed the least value on instructor characteristics; perhaps this reflects their level of confidence in themselves as learners due to their high degree of experience with the higher education process. Finally, it is interesting to note that female STEM instructors received a disproportionately higher number of comments for both strengths and suggestions. Is there something about the dynamic of female-taught STEM classes that elicits more open sharing from students? Or do students have preconceptions about women teaching these classes that lead them to think more "help" is necessary?
These are all possible questions for further research. We invite fellow instructors to join this dialogue and share their findings.