Student perspective about the impacts of feedback

Projects and Practices in Physics (P-Cubed) is a flipped, problem-based learning course for introductory, calculus-based mechanics. P-Cubed was designed using the communities of practice framework, with a principle learning goal to develop scientific practices. To promote students’ development of practices, students spend their in-class time working in groups of 4 to 5 members to solve complex physics problems. Scientific practice development is then facilitated through formative feedback and assessments aligned with the growth in these practice. Each student receives weekly, individualized feedback from their instructor (either a faculty member, teaching assistant or learning assistant) as a practice-based assessment. The feedback focuses on helping students improve their individual understanding and group collaboration through the development of key group-based practices such as decision making and planning. This paper reports on a series of semi-structured interviews that were conducted with students at various points in the semester to understand the impact feedback had on them.


I. INTRODUCTION
The learning environments of college level physics courses are being transformed resulting in more active, group-based environments [1][2][3][4][5][6] . In these transformed learning environments, students are typically required to change the ways they participate, which often includes a focus for students on developing their scientific practices (i.e. modeling, communication, group work, etc.) [5][6][7]. Homework, exams, and other summative assessments can provide useful feedback to students about their work in the course, but these mechanisms are rarely focused on the development of scientific practice. Additionally, summative assessment occurs after the learning takes place, which restricts its ability to help students develop skills [8,9]. This means that students need formative feedback to help them adapt and meet the learning goals of transformed classrooms [10].
By definition, formative feedback provides information to a learner with the intent of altering their behavior and thinking to improve their learning [8,11]. While beneficial for students, particularly in developing skills, formative feedback is not always straight forward to integrate into a learning environment [12]. For example, Hattie and Temperley highlight the importance of delivering feedback in a timely manner and the importance of the classroom climate in which the feedback is disseminated, received, and used by learners [13].
The Projects and Practices in Physics (P-Cubed) class is an introductory mechanics course at Michigan State University, designed to afford timely feedback in a positive climate [5]. As a part of P-Cubed, students are given individualized feedback from members of the instructional team on a weekly basis, which supports students in developing and enhancing group-based practices aligned with the learning goals of the course. The formative feedback is focused on each student's individual understanding and their contributions to their group. Within the broad categories of individual understanding and group work, students are given specific feedback about the development of scientific practices (e.g. constructing explanations in a group or how to engage in constructive argumentation). Previous studies within P-Cubed have examined the structure of the feedback mechanism and how giving feedback has impacted learning assistants in the course [10,14]. The next step is to investigate how the feedback impacts the students in P-Cubed. As the initial phase of this work, we present a case study of Carol and ask the research question: how does feedback impact Carol over her time as a student in P-Cubed?

II. CONTEXT & METHODS
Projects and Practices in Physics (P-Cubed) is a flipped section of introductory, calculus-based mechanics at Michigan State University (MSU). The course was designed to implement a problem-based learning approach [15], where small groups of students negotiate meaning, develop a plan and approach to solving complex physics problems, and utilize scientific practices to successfully complete their projects [5]. This work is facilitated by a tutor (either a faculty member, graduate teaching assistant [TA], or undergraduate learning assistant), who guides students, observes their work, and asks questions during class. Each tutor works with two groups of 4-5 students and attends mandatory training at the beginning of each semester. For more training and course details, see references [5,10,14].
To help in-class student groups improve the use of scientific practices, like group work or developing models, tutors write weekly feedback that is individualized for each student. The feedback is based on the tutor's in-class observations from the week, and students receive the feedback prior to starting new projects the next week. Formative feedback also serves as an in-class assessment making up 20% of their overall course grade. The feedback consists of three different categories [10]: (1) how well does a student develop their own physics understanding (individual understanding) (2) how well does a student ensure that all members in the group develop a physics understanding (group understanding)? and (3) how well does the student contribute to the group's management of their discussion and use of ideas (group focus)?
Focusing on these three different categories, the tutor provides two paragraphs to each student each week, where the feedback is split between feedback for the group as a whole ("group feedback") and feedback for the individual student ("individual feedback"). Each student in the group receives the same group feedback, which is targeted at helping the group improve their group-oriented practices and dynamics. In the group feedback, the tutor includes one thing the group excelled at, one thing the group needs to improve on next time, and provides strategies for achieving this improvement. The individual feedback includes the same positive, improvement and strategy framework described for the group but for each individual student. A part of training each semester focuses explicitly on writing feedback.
To study the feedback mechanism, two rounds of interviews were conducted with ten students in P-Cubed during the Fall 2015 semester by the last author, who was part of the curriculum design team for P-Cubed but not teaching during this semester. The first set of interviews were performed during week four of the semester while the second set of interviews were performed near the end of the semester. Due to the class structure between the first interview and the second interview, students changed groups twice in this time. The interview protocol was broken into three distinct sections. The first section was semi-structured [16] and focused on students describing their experiences in the class so far, specifically in their group. It asked them to describe their typical interactions in the class, how they thought their group was doing and their role in the group. The second section was more directed at feedback and explicitly asked them to describe their understanding of the assessment within P-Cubed with a specific emphasis on the feedback mechanism. They were asked to describe the type of feedback they received, why they thought they had received it and what the purpose of the feedback as an assessment tool was overall. The final section was a form of stimulated recall where the students were asked to read their previous feedback and then reflect on the impact it had on them in class [17]. In the first set of interviews, students were given their feedback from weeks 1-3 while in the second set of interviews, students were given their feedback from weeks 9-13 to read. The data for this study uses both the feedback students receive from their tutor and the student interviews. All names have been changed, and the quotes have been edited to remove pauses and "umms." In this analysis, we focus solely on the interviews and feedback of Carol, a 19 year old, Black female majoring in engineering. She is a sophomore and previously took high school physics. Carol decided to attend MSU due to available family support, her sister attending here previously, and the scholarship money she received. Carol was chosen as the sole focus of this study because she experiences a transformation in her role within her group in the classroom. This transformation was influenced by the feedback and, in the interview, she discusses how this transformation is related to the feedback supporting her in the development and use of scientific practices. As a pilot study, this is the first of several interviews that we hope to investigate to determine the impacts of feedback on all of the students in Carol's group.
This research aligns with using an explanatory case study approach due to the limited studies of the impact of formative feedback on scientific practices within the physics classroom [18,19]. In this analysis, Carol was chosen as a critical case since her interviews demonstrated some amount of change (though we do not expect hers to be a representative case). A case study approach aligns with this analysis because the boundaries between the feedback's impact and the impact of the rest of Carol's experiences as a student in P-Cubed are not always evident [20].

III. RESULTS
In this section, we describe the two main impacts the feedback had on Carol: (1) Carol advocating for her own ideas (first interview) and (2) Carol taking on a leadership role in her group by promoting scientific practices (second interview). At the beginning of the semester, Carol discusses the influences of feedback on her participation within the group. When asked to reflect on how it made her feel about her role in the group, she responded: "I think the feedback was trying to help me step up within the group and I know that is something I need to learn how to do myself. Because I usually just take on a side role in the beginning" -Carol, first interview This quote shows that feedback helps Carol think about the role she is playing in her group, particularly how she is contributing to her group. Carol indicates that the feedback is highlighting a problem that she believes she needs to address: play a more active role from the outset of the group. After Carol read her first week of feedback during the interview, she provided her take on how it affected her participation in her group: "...we all kind of had a different way of starting the problem out and I was perfectly fine with going off what...Logan [group member] said, as opposed to, I didn't know my idea was right. But so after I read this comment, alright for the next problem, I don't remember what it was, but maybe we can try this, this, and that." -Carol, first interview The above quote illustrates how in the first week Carol was not advocating for her ideas, but instead listening to others and following their approaches. She was hesitant to advocate for her ideas because she was not sure her ideas were correct. The feedback helps Carol by positively reinforcing her ideas, indicated by her saying, "I didn't know my idea was right," which implies that the tutor has highlighted that she does have correct ideas in the feedback. This positive reinforcement helps persuade Carol to advocate for her ideas as evidenced by Carol saying, "for the next problem...maybe we can try this, this, and that." This is an indication that she decided to speak up about possible approaches to try as a result of her first week of feedback. Encouraged by the tutor feedback, Carol also starts performing certain tasks to help herself address the feedback: "In order to prepare I suppose for the upcoming weeks, I would make sure I took the example problems or the equations I thought were useful, and made sure I have them there ready to share with the group. So I did try to build off of what Trejo [her tutor] said to do." -Carol, first interview She demonstrates how she is changing her behavior outside of class to prepare herself and to be able to communicate more of her ideas in class. Carol also makes a connection between the preparation of materials and advocacy of her ideas in a comment just prior to above quote where she says, "I didn't have the actual equations and concepts written down, so I could better advocate my ideas." To her, this preparation approach is in part addressing the feedback from Trejo (her tutor) that suggested Carol should advocate for her ideas. Carol looking over more of the material and preparing to use it in class is an example of her working to develop her individual understanding of physics. Furthermore, she says that she wanted to "have them there ready to share with the group," which shows that she views her notes as a tool to help her group in class. When responding to a question regarding if there were deliberate actions she took within the classroom to address the feedback, Carol expresses how she spoke up more and gives an example: "I think most of my feedback is 'go in and try to share what you know.' So, I definitely tried, and I would try to help make sure like Sue [group member] knew how to take this integral. So, I would definitely try to take whatever line it was [from the feedback] and say alright I need to share my idea. I know what to do so I am going to pause the group and say 'I think if we do that, then we can get it done'." -Carol, first interview Here Carol summarizes the change her tutor wants to see, which is sharing your thoughts and ideas, and the way she goes about trying to address the feedback. To Carol, helping group members, like Sue, understand problem solving techniques is one way of sharing what she knows. Carol is highlighting useful strategies of communicating in a group, namely ensuring everyone is on the same page (by saying "I'm going to pause the group") and ensuring everyone is learning or understanding the material (by saying "I would try to help make sure like Sue knew how to take this integral"). This extract also highlights a transformation in Carol's approach to being heard. She is now willing to stop the group in order to communicate her ideas to everyone. The positive reinforcement of Carol's ideas from the feedback have contributed to getting her to the point where she is affirming "I know what to do" and then pauses the group in order to make sure that is heard. This is the first indication from her first interview that Carol is starting to step into a more central role within her group.
By the end of the semester, we see Carol's progression of how she was stepping into a more central leadership role within her group. This "TA role," as described by Carol and her tutor, encourages Carol to use scientific practices that had not been described in the previous interview. Initially, Carol discusses how she has changed her approach to the course via the feedback by saying: "Well I thought it [the feedback] kind of helped me to expand the critical analysis part of what we actually have to do. And so again just changing the way I approach problems and starting with the system and then finding what is within that system and setting up a before and after equation. Like that's just ingrained into my thinking now. Like I am going to work through step A, B, C, D as opposed to going C, A, D." -Carol, second interview In this quote, Carol discusses the relationship between the critical analysis and scientific practices she is using to solve problems. She starts by saying the feedback helped her to expand her critical thinking and then describes what this critical thinking looks like to her. To her, the critical thinking looks like working through the problem systematically going from "step A, B, C, D as opposed to going C, A, D." Carol further illustrates what critical thinking looks like by describing a scientific practice, developing and planning a model, she is using indicated by her saying, "starting with the system and then finding what is within that system." Carol's explanation of how she has changed her approach to problem solving can be compared to a piece of feedback she received from Trejo in week 9: "I also noticed you had some excellent critiques about the plan and process on Thursday's problem where you pushed for more discussion about what the group was solving for and why the different analyses that Deirdre and Logan [group members] had done were not incompatible. I really would like to see you take up that rolecritiquing the process and ensuring y'all are on the same path." -Feedback from Trejo to Carol, Week 9 In the feedback, Trejo further highlights the scientific practices Carol is using to help her group build solutions. Trejo begins by highlighting Carol's ability to plan and carry out investigations. He says, "you had some excellent critiques about the plan and process," which demonstrates that Carol is considering what the plan is and how to proceed in creating a solution for their group problem. He continues by saying, "you pushed for more discussion about what the group was solving for," which shows that Carol is considering the outcomes of the group's model and is pushing her group to consider those as well. This demonstrates Carol's growth in the scientific practice of developing and planning models. Trejo also states how Carol encourages critical discussion in her group about the different analyses (proposed by two group members), focusing the group to consider their plan for their investigation. Trejo ends the feedback by encouraging Carol to take up this role of critiquing and ensuring the group is working together. Carol's quote and the week 9 feedback demonstrate how Carol has moved beyond just trying to advocate her ideas and has started to become more of a central member of group. She is using scientific practices to help her group critically think about their solutions to complex physics problems. The idea of expanding Carol's role within her group from an advocate role to a tutor-like role was suggested in week 11 feedback: ...on Tuesday, your critical questioning really helped the group defend and explain its choice of system and resulting physics...I think your group really benefits from your critical eye... For this next week, I'd like to see you broaden that critical role to include more physics questioning of your group. That is, I'd like to see you include a tutoring role in your repertoire. You know the physics well, so asking a few of your own tutor-like questions to make sure all of your group members are developing their understanding would be a nice addition to your current strengths. -Feedback from Trejo to Carol, Week 11 In this feedback, Carol has been complimented on her ability to ask her group "critical" questions about their solutions, which helps her group develop the practice of constructing and defending their explanations. Here, Trejo is recommend-ing that Carol should broaden this "critical role" to a "tutoring role." Before having re-read her feedback in the second interview, Carol was asked whether her TAs had asked her to improve on a new thing after she had made improvements on something else. In response, Carol describes what this "critical role" looks like to her and how this relates to a TA role: "...[I would] start with the critical thinking and then also making sure I asked or pushed my group to also more honestly focus on critical thinking, but with kind of the TA role kind of again. Focus on making sure that everyone else is on the same page and can articulate the physics behind it." -Carol, second interview Here Carol discusses the relationship between a "TA role" and the critical role. She demonstrates this by saying, "[I] pushed my group to also more honestly focus on critical thinking, but with kind of the TA role kind of again." She even discusses how she is trying to get her group to construct explanations and work together when she says, "focus on making sure that everyone else is on the same page and can articulate the physics." Relating this back to Carol's week 11 feedback, Trejo asked her to start asking these "tutor-like" questions to support her group in physics understanding. We see that Carol actually takes up this teaching role in the above quote. Carol goes on to describe what these "tutor-like" questions look like and how she leads her group through figuring out problems: "I was like 'wait, stop, run that back, what's our system, stop,' and I was like 'let's focus on the relationship because so we know what's actually present in this first state and what's present in the second state'. I guess that kind of ties in as well to questioning and that whole aspect of the critical thinking." -Carol, second interview This quote demonstrates how Carol is participating within her group by the end of the semester. First, Carol is showing leadership because she takes control of the group's conversation by saying, "wait, stop, run that back..." followed by asking a series of questions to her group. The first question Carol asks is "what's our system," which shows Carol is encouraging her group to think about the scientific practicedeveloping and defining their system. Then Carol shows leadership again by stopping the group and pulling their attention to relationships within the problem. She identifies that this will help them relate "what's present in this first state and what's present in the second state." This shows that she is using the relationship to develop a plan and bring together different parts of the problem. Furthermore, Carol recognizes that she is using these scientific practices in helping her take on a "TA role" because she says, "that kind of ties in as well to questioning and that whole aspect of the critical thinking." By the end of the semester, we see Carol building on her feedback to encourage these scientific practices in her group and to take on a leadership role.

IV. DISCUSSION
Returning to the research question, we see that, the feedback has encouraged and supported Carol in changing her participation within her groups. In the beginning of the semester, Carol struggled with having a voice in her group. By the end of the semester, Carol was not only advocating for her ideas, but using her voice to encourage her group to think critically and to guide them through complex problems by assuming a teaching role within her group. Initially, the feedback empowered Carol by pointing out when she had the right idea. By providing positive reinforcement and encouragement, the feedback led Carol to share her ideas and advocate for them. The feedback continued empowering Carol later in the semester by acknowledging her use of scientific practices, encouraging and supporting Carol in the use and development of those practices, and providing suggestions on how Carol could use these practices to help her group grow by taking on a "TA" role. By the end of the semester, the feedback has changed from encouraging Carol to advocate for her ideas to encouraging Carol to take on a "tutor-like" role, transitioning from a group member to taking on more of a leadership role within her group.
One limitation of this work is that the feedback is only one of the influences in this class. Therefore, we can claim the change in Carol's group participation is only partially due to the intervention of her feedback. For example, the feedback captures some of the interactions between Carol and her tutor, but it does not capture all of the interactions between her and her tutor, how Carol's group responded to her change in participation, or how other members of her group changed as a result of Carol. Carol also changed groups between the first and second interview and this study does not account for these changes in these group dynamics. To capture how Carol interacted within class, we would need to analyze in-class video data, which would allow for the aforementioned interactions to be studied and for an analysis of the similarities and differences between Carol's perspective of what happened versus what happened in the moment. In-class video data would further inform Carol's case study by looking at how the feedback is enacted within the class; however, Carol is also only one case to analyze, and she is not fully representative of all student experiences with feedback in P-Cubed. For instance, this paper did not attend to how Carol's gender and racial identities influenced her in-class or experiences with the feedback. In future work, we hope to use the interviews, feedback, and in-class data of Carol and the other interviewed students to address these limitations and further explore the impacts of feedback.