Understanding Student Preparation Of Exam Note Sheets

In a previous study, I allowed introductory physics students to create a notecard (or sheet) for their midterm and final exams in an attempt to remove equation memorizing as a focus of the course. I hoped to use the study of these cards as an epistemological lens that would uncover their perceptions and attitudes about the course. Without follow-up questions, though, epistemology remained unclear. I have continued this line of research by adding anonymous survey questions that probe why students chose to include what they did, how (if at all) the notes were helpful, and how their card preparation changed throughout the semester. Through my analysis, I found cases where the survey questions reveal epistemological insight about students the note sheets alone would not. For example, one student with an equationcentered note sheet did not see equations as the most central course component, but he considered himself fluent enough in concepts that he felt he could leave conceptual statements out. I also discuss some of the more thoughtful survey answers I received, some future study possibilities, and efforts to discuss exam preparation in the classroom.


INTRODUCTION
Previous research has used methods such as interviews [1] or multiple-choice surveys [2] to gain insight into student epistemologies (defined here as context-dependent beliefs about the nature of science and learning).This study is an extension of previous work [3] where I tried to uncover student epistemological thinking by studying the note sheets students prepare for their final exams.
In that work, I allowed students in two introductory mechanics courses to bring a 4" x 6" notecard for the midterm and one side of an 8.5" x 11" sheet for the final exam.I sought to reduce emphasis on equation memorizing as well as answer the following questions: (1) In what different ways do different students choose to use the allowed note space?(2) Can we find associations between what students choose to include and their exam performance?(3) What can student notecards tell us about what the students see as important in the course and what might be examined?
I classified what students included on the sheets with minimal ambiguity.The majority of student exam notes were classified as equations, but other prevalent categories included definitions, units, physical constants, conceptual statements, pictures and diagrams, and guides to symbols.Though my data sets were too small for rigorously mathematical conclusions, I found little relationship between the number of notes on a note sheet and the final grade; specifically, two note sheets I analyzed in detail from opposite sides of the grade spectrum were quite similar in raw number of notes but different in organization.
Despite the fact that this study yielded rich and interesting data, it was limited in epistemological insight simply because it allowed for only inferences at best.For example, a student who chose to include mostly equations on her note sheet (as was the case frequently) may have seen equations as the only important things in the whole class or the only items worth committing to paper.I also could not say whether students mindlessly commit notes to paper or judiciously choose what to include or leave out.
As indicated in the conclusion of my previous work, I decided to address some of these questions.I incorporated an anonymous survey into my collection of the students' note sheets with the goal of getting more direct information from students regarding their preparation of these sheets.
Inspired by feedback I received on my previous work, the questions I asked were: (1) How did you decide what to write on your note sheet?(2) How was either writing or having the note sheet helpful to you, if at all? (3 -for the final only) Was your preparation of notes for this [the final] exam different than it was for the midterm?How?
As with last year's data, student note sheets were heavily equation-centered.I will document the survey results and focus on cases where students either had unique takes on the "equation sheet" strategy or explicitly focused on something besides equations.science.Our two courses focused on mechanics are Physics for Filmmakers [4] (PfF) and Fundamentals of Physics I (FoP).These courses provide the present study's population.
Both courses in this study are algebra-based.PfF uses clips from action and science fiction movies to motivate discussion about kinematics, dynamics, collisions, energy, and other mechanics topics.Labs are often filmed and analyzed using Tracker [5] software.Most of the students in this class come from the Film and Video department.The goal of the course is to help them make more deliberate choices when depicting physics in their artistic work.Students must complete a final project where they demonstrate correct physics in a film they create.The Spring 2013 version of the course started with seventeen students, with fourteen completed the final exam and eleven completing the note sheet with survey questions.
FoP has more traditional mechanics course coverage and is taken primarily by Acoustics students in our new Bachelor of Science program.The Spring 2013 version of the course had seventeen students.Course activities included lectures, labs (some in common with PfF), tutorials, and demo-centered discussions.Ten students completed the survey.
FoP included more mathematics, including a small amount of trigonometry which was not a PfF prerequisite.Many FoP students are either enrolled in or are planning to take calculus.There was still noticeable overlap between the final exams for the two courses, especially on conceptual questions.

Preparation and Collection
For both courses, students were informed well in advance that they would be allowed to bring notes to the exam, and they were told they could use the paper space in any way they wished.Students were also informed that the final exam would consist of multiple sections.Both courses' exams featured two parts: a conceptual section containing multiple choice items, short answer problems, and graphing representations, and a more mathematical section consisting of written problems.The FoP exam was more conceptually and mathematically involved, so those students received a three page guide listing topics and sample problems.
Because I was the course instructor, participation in the survey was kept optional and anonymous.All students who consented to my survey questions prepared notes, but not all spent a long time doing so.Students left their names off the note sheets and attached them to their survey responses.Students retrieved their midterm exam notecards later, but I returned neither the exam nor note sheet from the final.

Observations Regarding Final Exam Survey Questions -Spring 2013
As in my previous study, a large portion of the note sheets turned in were devoted to equations.Every note sheet included a number of them with the exception of a single PfF note sheet with only three equations on it.This trend was not a surprise, but I checked to see how many students explicitly mentioned writing down as many formulas as possible as well as why.Eight of the eleven FoP students and seven of ten PfF students prominently mentioned writing out formulas as part of the note-writing process.
Students mentioned using other course resources less often when constructing their note sheets.Among the ten students studied in the Spring 2013 Physics for Filmmakers course, two of the students said they referred to old quizzes, four reported using their notes (either online or from their notebooks), and one used his or her textbook as a reference for the note sheet.Among the eleven Spring 2013 FoP students I surveyed, five used their notes (either online or from their notebooks), one mentioned looking through old homework, and one student, who I will codename "Jordan," explicitly brought up conceptual material.Specifically, Jordan said, "I wanted to get as many concepts as possible with some examples.I followed the concepts / practice problem sheet you gave us as a guideline."In the next section, I will compare Jordan's note sheet with those of students who self-reported their preparation as more equation-centered.
The final survey question asked if their note preparation was different between the final exam and the midterm exam.All but two students in the PfF population and Jordan in FoP indicated the process was similar, and three students in each group said the main difference was simply the amount of material to be covered on the exam.In the Filmmakers population, only one student gave a more nuanced answer: "I knew a little better what to expect so I didn't put ALL the variations of the equations like I did last time.Just the main ones."The other Filmmakers outlier was the student who barely prepared anything at all.Representing the FoP population, Jordan indicated, "I studied more of the concepts that I didn't understand more so than I did before."

A Comparison Of Relevant Note Sheets
In my previous work, I did a detailed comparison of two note sheets from students who both wrote a large amount of material but performed differently on the exam.I cannot do an exam comparison in the present study since the surveys were anonymous.However, I will compare the sheets from three Spring 2013 FoP students where the students approached the note sheet task from different epistemological angles.
One student I will look at in detail is Jordan, described above.Another is "Georgie," a student who tersely noted that he "wrote just equations," and "Having formulas helps a lot."Finally, I will look at "Dana," a student who mentioned both concepts and equations in his survey.Dana's wrote, "I knew concepts really well, so all I really needed was a list of equations I would need."I will break down the content of these three sheets and discuss their organization.
In addition to Jordan's response mentioned above, the other two of Jordan's survey questions were concept-centered, including, "I was able to use a lot of what I wrote to help with concepts on the test," and "I studied more of the concepts that I didn't understand."Not only was Jordan's note sheet the lengthiest I received this semester, it was most balanced in content and approach.
Jordan's sheet featured over seventy equations, but also at least fifteen detailed conceptual statements, an equal number of diagrams, including a number of translations between position, velocity, and acceleration graphs, ten fully worked examples, and a number of definitions, unit guides, and physical constants.In my past work, I've seen sheets as lengthy and well-prepared, but the focus of Jordan's survey answers is interesting.
Without seeing Jordan's survey, I would not have anticipated such a conceptcentered attitude toward the note sheet given how prevalent the equations and mathematics were, despite a larger-than-average focus on concepts.
Georgie wrote a lengthy and well-organized equation sheet for notes.There were over forty different equations, and five definitions written entirely in equation form.To say that Georgie only wrote equations is not entirely correct, but much of the extra writing was equation-centered also.There were twenty separate symbol guides that explained the equations' terms.These equations and symbol guides was just about the sum of it; I coded only two statements as conceptual: one stating energy conservation and one explaining the role of the velocity vector in the expression for angular momentum of a free particle.Finally, Georgie included the value of G and the SI unit breakdown of a Newton.Words above groups of equations indicated general subject area, like "Gravity," "Energy," or "Centripetal." Dana's sheet, as indicated, was also extremely equation-centered -in fact, more so than Georgie's.The sheet included fourteen equations and nothing else.There were no guides to symbols, pictures, examples, conceptual statements, or even equation-like definitions from kinematics.In the survey, Dana indicated that this list "was a little helpful.I forgot a few things I should've written down."My best guess is that if Dana had written more, it would have been equations relating to different concepts.Many topics covered on the exam were neglected.Without the survey, I might have assumed Dana was simply focused on equations and poorly-prepared, but his survey response indicated he felt comfortable with the concepts.I could not have known that fact about Dana's self-efficacy from the note sheet alone.

More Reflective Survey Responses From Midterm Exams
The midterm and final exam surveys occasionally gave different types of student responses.In Fall 2012 and Spring 2013, students seemed more willing to reflect more after the midterm than after the final.I received more thoughtful epistemological responses to my survey items, likely due to the fact that the exam conditions were less stressful.
The most recent final exam survey answers in FoP, for example, tended to be terse one or two sentence answers.The lengthiest response not belonging to Jordan said, "I thought the most important thing would be to be able to solve all the equations[;] I also decided to have some SI units and Kepler's laws."While some of my midterm responses were that length and depth, other students from the Fall 2012 class were more reflective, such as one who said, "I looked through each homework, lab, and quiz and thought I had a good enough understanding of the concepts that all I needed to know was the formulas.Although after going through the test I realized I didn't know this stuff well enough conceptually, and so all my equations look meaningless."Another wrote, "Probably should have included some homework examples, but I didn't, regretting that now.I mostly studied by reviewing homework, but I seem to have glossed over some things, thinking I understood them better than I do."These answers were not only more thoughtfully written out than my final exam surveys in general, they were more reflective about the studying process.
One was even more explicitly epistemological.His (or her) notes were centered "mostly around formulas and example problems.I did this not to re-'replicate' them, but to have some sort of step-by-step basis and understanding for long answer problems.I knew multiple choice/graphing were to be based on true knowledge."Having students articulate that more conceptual-styled problems touch on "truer" knowledge is a benefit to this survey methodology one cannot get from a simple analysis of note contents.

Class Discussions
One hope of mine in giving the survey at the midterm was for students to be able to modify or improve their performance on the final.I thought that if given a chance to be reflective on their note-writing process, students would be inspired to discuss that process after the fact and come up with ideas they would use on the final.This past semester, that did not happen much with either class.Though I devoted some time in both classes just before the final to reviewing content and exam composition, the students did not explicitly take it as a chance to improve their note sheets.
There are a few steps I can take in future semesters to improve this.One is to weave thinking about note sheets into homework.My students are always willing to discuss homework questions at the start of class, so perhaps a writing prompt before or after the midterm would spark a discussion about useful study strategies.A prominent benefit cited by students in making these sheets is that preparing the sheets helps them to study.In fact, I received feedback on an earlier presentation of this work indicating that the students would be better off if their lists of notes were explicitly called "study sheets" rather than "note sheets" or "cheat sheets."

Possible Study Extensions
It is likely that a semi-structured interview starting with my written survey questions may be the best way to get detailed responses from students.As course instructor, I had ethical concerns interviewing my own students, but I would be in favor of using interviews for a population outside my personal courses.On a written survey, even the most thoughtful students will only write a sentence or two, and even a short interview could indeed prove richer in comparison.
Another option one could pursue to distinguish between equation-centered students and conceptuallyfocused ones is to provide a common list of equations for all students and give them the ability to add to that list.Since this method could deprive students of the benefit of writing their own equations, I will not likely pursue this modification in the near future.

SUMMARY AND CONCLUSIONS
I began work on studying student exam note sheets not simply from a desire to review the contents, but to investigate what we can learn about student epistemologies from what they choose to include in their allotted space.Previous work including mostly note sheet contents allowed only the most superficial inferences, so the present study added survey questions that asked students to reflect about what they felt was important to include.For the most part, as anticipated, the students felt equations were the most important thing to focus on in preparing the note sheets.
However, there were examples found in the data where students self-reported insights into their epistemological thinking that would have been difficult to anticipate from the note sheets alone.Jordan's outlook on the survey questions was centered on concepts, despite the mathematical focus on much of that note sheet.
Dana gave the most minimal, equation-centered sheet I received, but this was not due to an equation-centered epistemology necessarily; conceptual self-efficacy was evident.Georgie's note sheet matched closely with the corresponding survey questions.In summary, epistemological insights I have gained from this study come from directly asking relevant epistemological questions to students.The classification of note sheet contents helped point to pieces of data that would reward deeper study, but it did not serve as an epistemological study in itself.
The prior semester's surveys occasionally gave far more reflective survey responses.Ideally, I would like to see similar depth from everyone either in a class discussion or on his or her survey.Decisions regarding how to get the class to discuss their exam preparation remain; I could either incorporate epistemologyfocused prompts into the homework or devote more time to class discussion.Though many students saw the note sheet writing as useful, it is an open question as to what setup of these tasks would help them both understand the material best and sympathize with my main epistemological goal for the courses: that students should see physics understanding as consisting of multiple types of thinking and representation.