Examining student growth in laboratory notebook practices in introductory physics courses

Laboratory notebooks serve as the primary records for research experiments, demonstrate how researchers went about their work, and tabulate the data collected in the experiment. Recent undergraduate laboratory recommendations emphasize the importance of helping students develop their scientific documentations skills throughout the curriculum. Learning proper lab notebook practices is important for students in preparation for graduate school and the STEM workforce. Laboratory notebooks can also be a valuable tool for instructors to assess a student’s understanding of lab activities and the underlying physics principles. This study analyzes shifts in student understanding of standard notebook practices during the first introductory physics lab course. This research was conducted through a series of preand post-surveys given in the introductory physics labs. The surveys cover how students report using lab notebooks, how they learned notebook practices, and their attitudes toward and beliefs about using lab notebooks. This paper examines student growth in specific, self-reported laboratory notebook practices and reviews the instructional methods and feedback that were most effective for first semester physics students.


I. INTRODUCTION
Laboratory notebooks provide a critical record of everything done and observed and the results obtained during an experiment. They also help researchers 15 prepare presentations and papers to disseminate their findings, and they can be shared with other researchers to repeat or advance experiments. Thorough documentation is key for good communication between researchers and from researchers to the public. To prepare students for graduate school or a career in a STEM field, it is important that lab curricula emphasize developing authentic scientific documentation skills.
There has been growing interest in laboratory notebook use and practices in undergraduate physics labs in the last decade. A handful of organizations have published recommendations for laboratory reform, which include developing lab notebook skills as an authentic form of scientific communication. The American Association of Physics Teachers (AAPT) recommended specific standards for students to meet in introductory and advanced laboratory courses in physics in order to be ready for a career in a STEM field or for graduate school. The AAPT Recommendations for the Undergraduate Physics Laboratory Curriculum calls for an increased focus on developing proper notebook practices in introductory physics labs. These recommendations identify using lab notebooks as a technical and practical laboratory skill and as an integral part of communicating physics [1]. The Phys21 report from the American Physical Society reinforces that professional communication is an important skill for students' career readiness [2].
These ideas about the importance of developing laboratory notebook skills are not new, however. In 1962, Irving Dayton from Swarthmore College published an article about the value lab notebooks can play in these courses [3]. Dayton discusses how focusing lab curriculum on the notebook has a positive effect on the students' understanding of physics. He also mentions that replacing formal lab reports with notebooks gives the students more time to focus on the physics and gives the instructor a better understanding of what each student knows. Recent studies show that evaluating lab notebooks are indeed an effective way to assess students in the lab [4][5][6][7]. While rubrics often provide guidelines on the content that students should enter in a laboratory notebook, too much structure in the lab notebook format can turn a notebook into a lab report instead of a real-time record of work in the lab. The goal should be for students to develop authentic documentation and communication practices [8].
Lab notebook skills are not only important for the classroom, but they are vital for students pursuing careers in research. Researchers use similar lab notebook practices whether in an educational institution, government facility, or private industry; lab notebook practices are highly transferable. A recent study shows that many incoming physics graduate students do not have the scientific documentation skills needed for graduate level research [9]. The study reported two common themes from these students when they were sur-veyed about their lab notebook experiences. First, the students were not assessed in their undergraduate courses with the notebooks as an assessment tool. And second, the graduate students learned the practices they use to maintain their graduate research notebook though hands-on practice or what the researchers call, "authentic lab experience." Students with exposure to this type of lab or to research gain practice in many important skills. Some organizations propose all introductory laboratory courses shift from standards-based to hands-on, discovery-style labs for the students [10].
Discovery-style or inquiry-based labs give students an opportunity to practice many important lab skills including scientific documentation [2]. Atkins indicates that using authentic forms of scientific writing, including laboratory notebooks, in undergraduate physics labs is "not only an outcome of scientific inquiry but a way of performing scientific inquiry" [5]. However, these labs also allow students to develop bad habits if not guided properly. Researchers at the University of Colorado-Boulder examined the most important features in notebooks of their graduate students and discussed how to include them in a classroom [8]. The researchers' recommendations explain how to organize the notebook and what type of information is important to include. While these recommendations are for using lab notebooks in an upperdivision physics lab, they can be adapted for use in the introductory lab.
Since introductory physics lab students have widely varying experiences with lab notebooks in high school or other college lab classes, we developed a survey that we used to examine the change in students' practices, beliefs about, and attitudes toward lab notebooks from the beginning to the end of their first semester lab course and in subsequent lab courses in the introductory physics sequence and beyond. This study looks only at self-reported changes in laboratory notebook practices during the first introductory physics lab and the instructional methods and types of feedback the students found most useful.

A. Data Collection
For this study, we developed surveys, a "pre-survey" and a "post-survey," that we gave at the beginning and end of the semester, respectively. These surveys consisted of the same 31 Likert-style questions answered on a 5-point scale from strongly disagree to strongly agree. The first 10 items ask about lab notebook uses and practices in general; whereas, the remaining 21 items ask students about their own attitudes, beliefs, and practices. The pre-lab survey also included questions regarding demographic information and previous experience using lab notebooks. Students who indicate that they have never used a lab notebook in a prior lab course in high school or college only respond to the first ten items on the pre-lab survey. The post-lab survey contains 5 ad-TABLE I. Survey questions related to lab notebook practices on the pre-and post-survey.
Q1 A lab notebook should look like a lab report with an introduction/ Q12 I often wait until the end of an experiment or lab period to purpose, materials and methods, data, analysis, and conclusions in write in my lab notebook. that order.
Q21 I sometimes record what I think the outcome should be or what Q2 When a mistake is made it is acceptable to erase it or white it I think the professor is looking for instead of recording what I out and write over it.
observe or measure. Q4 Failed experiments, contradictory or "bad" data, and outlying Q23 I make notebook entries while I am conducting an experiment. points should be recorded in the notebook during an experiment.
Q25 I always sign and date my notebook entries the day I make them. Q6 It is important to sign and date lab notebook entries.
Q26 I usually write observations, comments, or conclusions about Q7 It is never acceptable to remove or replace data collected in a my procedures, data, analysis, and results in my lab notebook. lab notebook, even if you think the data are "bad" or not useful.
Q27 I sometimes copy material from my partners' notebooks Q8 Supplementary data and graphs should be pasted/taped into a into mine without ensuring that I understand the entries. lab notebook.
ditional Likert-style questions and 7 short-answer questions about notebook-related experiences that semester. The extra Likert-style questions pertain to lab notebook instruction and evaluation in the class, and the short-answer questions ask students about the types and perceived usefulness of the training and of the feedback that they received on their notebooks throughout the semester. When students took the surveys, they used unique identification numbers on them so that their pre-and post-surveys could be compared over the course of the semester, and ultimately to examine longitudinal growth in notebook practices through the course of the laboratory curriculum.
This study focuses specifically on the 13 survey items shown in Table I, which are related to laboratory notebook practices on the pre-and post-surveys. The numbering (Q1, Q2, ...) refers to the item numbering on the survey. The study also examines students' short answer responses on the postsurvey about what kind of notebook instruction and feedback they found most useful in their laboratory class.
We gave this survey to students in first semester calculusbased introductory physics laboratory sections at Lewis University, a private comprehensive Masters-granting institution in Illinois, from the fall semester of 2016 through the fall semester of 2017. This laboratory uses primarily inquirybased labs, many with an experimental design component. We collected a total of 159 surveys from 91 unique students. From these, there were 69 paired surveys in which students completed both the pre-and post-surveys. Of the students who participated in this study, 62% are planning to major in a physical science where lab notebooks will likely play an important role their future studies and careers. A majority of the other students were Mathematics, Computer Science, or Computer Engineering majors.
The onset of this study coincided with a departmental decision to increase the focus on lab notebooks at all levels. This occurred as a result anecdotal improvement in students' laboratory notebook skills when an instructor piloted enhanced use of lab notebooks and increased notebook instruction in an introductory lab section the previous year. Though we did not specify a common curriculum for teaching lab notebook practices, all instructors spent at least 30 minutes discussing similar lab notebook practices and expectations during the first lab class, using slides or handouts, whiteboard demonstrations, and notebook examples to varying degrees. All instructors provided feedback through comments in the students notebooks; however, one instructor also met with the students individually early in the semester to review their notebooks and provide verbal feedback. One instructor used a formal rubric for grading while the others used less structured point-based scoring methods. The variations of instructional and feedback methods paired with the short-answer questions on the post-survey gives some insight into which methods students found most beneficial.

B. Data Analysis
A mixed methods approach is taken in an attempt to correlate how students' reported lab notebook practices change over the course of a semester with the types of instruction and feedback they report as most useful.
Since the Likert data is ordinal and a normal distribution cannot be assumed, the Wilcoxon signed ranks test is used to analyze the quantitative data produced by the surveys. This analysis uses a paired data set created by comparing the presurvey responses to the post-survey responses of each student to look for fluctuations in the median difference in the responses from the null hypothesis, or no shift from the pre-to the post-survey [11,12]. A z-score of z > 1.96 indicates a confidence level of ≥ 95%, or p < 0.05. A positive z-score indicates an increase on the average Likert scale response; whereas, a negative z-score represents a decrease.
On the post survey, students were asked to provide written responses to several free response items. We did a thematic analysis of students' responses to two of these items related to what training/instruction students found most helpful and what feedback methods students found most beneficial. We used a generative coding approach for this analysis in which we read through the comments, noted emerging themes, and then went back and categorized the responses by like themes, refining and combining themes during the process of categorizing the responses [13]. When responses contained more than one theme, we categorized each distinct part of the response accordingly.
Finally, we ran the Wilcoxon signed ranks test on subgroups of student responses sorted by theme, which allowed us to identify if there were any shifts that could be tied to specific instructional or feedback methods.

A. Quantitative Analysis of Likert Data
The Wilcoxon signed ranks test analysis for matched pairs of pre-and post-surveys found significant shifts (p < 0.05) in four of the claims students made about their laboratory practices. These are shown in Table II. In the table, n is the number of paired surveys with different pre-and postlab responses since items with no shift are excluded from the Wilcoxon analysis. The shifts in items Q2, Q8, and Q25 all indicate a better understanding of notebook practices. However, while instructors anecdotally observed an improvement in students' notebooks over the course of the semester based on a general increase in notebook grades, a rigorous analysis of student notebooks would need to be conducted in order to determine whether there was actually a significant improvement in practice, not just in understanding. The shift in Q1 may indicate that students better understand what should go into a notebook; however, it may also indicate that students see the notebook having a formal structure like a lab report, which is not a shift toward more professional notebook practices. Further study is needed to understand this shift.
Though only four items showed significant change, students demonstrated knowledge of lab notebook practices for many of the other items. For example, 93% of students agreed or strongly agreed with item Q4 on the pre-survey compared with 96% on the post-survey. A large majority showed understanding of this practice in both the pre-and post-surveys so there was not a significant shift in their responses.
It is also important to note that when we ran the quantitative analysis on subsets of students grouped by instructor, any variations in instruction and feedback did not impact shifts in student responses significantly. Though these were low statistics cross-checks, the decision to combine all lab sections in order to do the analysis on a larger sample is reasonable.

B. Thematic analysis of short answer questions
In order to determine what training/instructional methods for keeping a lab notebook students found most beneficial, we grouped the short answer responses to the question "What training/instruction did you find most helpful in this lab section?" on the post-survey by theme. The six categories of common themes generated from students' responses are: class instruction, instructor feedback, handouts/rubrics, notebook purpose, and specific task training.
The "class instruction" category arose from responses that mention the formal instruction on how to keep a lab notebook in general and in specific labs, including showing examples of authentic lab notebooks. The "handouts/rubrics" category came from responses that discuss handouts or documents provided at the beginning of the semester or during the semester such as grading rubrics and notebook instruction slides. The "instructor feedback" category emerged from responses that mention the instructor making personalized corrections in students' lab notebooks, writing feedback on the rubric, or giving oral feedback in one-on-one meetings. The "notebook purpose" category came from responses that mention instruction on why scientists use lab notebooks in industry and research or on why keeping a lab notebook is important for the students, such as for preparing presentations and papers. The "specific task training" category developed from responses that discuss how to insert graphs into the lab notebook, how to properly sign and date entries, how to handle mistakes, and other practical details students sometimes struggle with when keeping a lab notebook.
It is clear from Table III that students find the rubrics and handouts very beneficial. They indicate that rubrics and handouts give them guidance on what should be included in the lab notebooks. One student wrote in their response, "The handout with a specific checklist of what needs to be in the pre-lab, the lab, and the summary was extremely beneficial, I used it for every lab." Another student benefited from "the rubric for grading the lab notebook which was very detailed and outlined described exactly what was needed in lab notebook." Though "instructor feedback" was not indicated as the most useful method of instruction for many students, the "instructor feedback" responses centered on a particular theme: students appreciated feedback that was specific to them and helped them improve their notebooks. One student said, "We were actually able to meet with him and get feedback, which was intimidating but also substantially beneficial." Another student talked about personalized feedback with a similar positive attitude: "Professor feedback helped me improve my  In another short answer question on the post-survey, students were asked, "What feedback on your lab notebook did you find most helpful in this lab section?" A similar thematic analysis was done on these responses, and the following categories emerged: improving notebook content, indicating mistakes or missed elements, neatness and notebook organization, and types of feedback.
The "improving notebook content" category emerged from responses about how feedback helped students improve certain sections of their notebook like data recording, data analysis, graphs and tables, and conclusions. The "indicating mistakes or missed elements" category includes a range of comments about the instructor pointing out mistakes or missing elements. The "neatness and notebook organization" category includes the instructor critiquing legibility, formatting, and other organizational practices for notebooks. The "types of feedback" category encompasses comments describing effective general types of feedback like oral feedback or writing comments on the rubric. Table IV shows that students find having missing elements and mistakes identified to be the most useful feedback. The wording of many of these responses indicates that students want to know where they lost points on the lab. One student found "circling things I missed or messed up on and writing in what I should have there" to be beneficial. Similar comments about grading also appeared in the "neatness and notebook organization" category. For example, a student wanted feedback on "the correct format a lab notebook should be kept to receive a good grade." While the comments in the previous categories focused more on improving notebook grades, the "improving notebook content" comments focused more on growth and improvement. One student found that "the ways on how to write up data analysis helped me tremendously," and another noted that instructor comments on "what data to look at for my conclusion" were beneficial.

IV. CONCLUSIONS
When we ran the Wilcoxon signed ranks analysis on subgroups of data sorted by theme, many of the shifts previously identified were observed in several subgroups. Also, many of these subgroups have small populations, which limits the effectiveness of this analysis. Although this initial study does not definitively link any of the quantitative shifts to specific instruction or feedback methods, it sets the stage for future investigations that can more deeply explore how certain types of instruction and feedback lead to shifts in understanding of laboratory notebook practices and in the practices themselves.
While these additional studies are needed, the quantitative and thematic analyses yielded interesting and useful results on their own regarding student understanding of notebook practices and beneficial instruction and feedback methods. Moreover, students found this additional emphasis on notebooks in the introductory lab to be beneficial. In fact, 85% of students indicated that they agreed or strongly agreed with the statement on the post-survey: "My lab notebook skills improved during the course of this semester." From a preliminary analysis of the survey data from the second and third semester introductory physics labs, we found that students do seem to build upon their experiences and understanding from prior semesters. However, the number of longitudinally paired surveys across multiple semesters is low so more data collection is needed. Nevertheless, scaffolded-instruction in notebook practices over multiple semesters will ultimately benefit the students and prepare them for STEM careers or graduate school.

ACKNOWLEDGMENTS
We would like to thank the Physics faculty at Lewis University for allowing us to collect data in their lab classes and the students in the introductory physics labs for participating in this study.