Examining students ’ personal epistemology : the role of physics experiments and relation with theory

We investigated students’ epistemological views of physics with a specific focus on physics laboratory work. The roles of experiments in physics have been underemphasized in previous epistemology research and there is a need for a broad view of epistemology that systematically incorporates experiments into discussions about the structure, methodology, and validity of scientific knowledge. An epistemological framework incorporating these features guided the development of an openended survey, which was administered to students in algebra-based and calculus-based introductory physics courses, an upper-division physics lab, as well as to physics Ph.D. students. We identified several differences in students’ epistemological views of physics. Regarding the roles of experiments in physics courses, introductory students viewed experiments as supplemental tools for conceptual learning and problem solving, while higher-level students valued unique things experiments provide, including experiments as opportunities to emphasize the empirical nature of physics and the development of scientific and professional skills. Regarding views on the relationship between theory and experiment, introductory students heavily emphasized experiment testing theory. However, higherlevel students recognized a holistic cyclical relationship: experiment inspires theory, theory guides experiment, experiment tests theory, and theory explains experiment.


I. INTRODUCTION
Epistemology is an area of philosophy concerned with the nature and justification of human knowledge and generally refers to the theory of knowledge, knowing and learning [1,2].In educational psychology, the dimensions of personal epistemology can be clustered into two central areas: the nature of knowledge (what one believes knowledge is) and the nature or process of knowing (how one comes to know) [3].Within physics education, students' personal epistemology includes their views about the nature of physics knowledge and learning physics [4].Personal epistemology held by individuals has been linked to their academic learning [5,6] and more attention has been given to students' epistemological development in physics and science education more broadly [7,8].
In the past few decades, researchers have measured students' personal epistemology about science and learning science [9][10][11].There are two main foci of previous epistemology assessment tools in physics and science education.One focus is students' views about learning physics, especially problem solving and conceptual understanding in physics, which includes the Views About Science Survey (VASS) [10], the Maryland Physics Expectations Survey (MPEX) [11], and the Colorado Learning Attitudes about Science Survey [6].The other research focus is students' views about the nature of science, which includes the suite of the Views of Nature of Science (VNOS) surveys [5].Across all of these surveys, there are very few items focused on experimental aspects of physics (e.g., experimental design, measurements, and uncertainty).In recent years, there has been increased effort to assess students' views about learning in physics lab courses via the Colorado Learning Attitudes about Science Survey for Experimental Physics (E-CLASS) [12,13].
The neglect of experiments and lab courses from much prior epistemology research is surprising because theory and experiment are twin pillars of physics and they provide two sources of knowledge: one in the form of modelbased reasoning and the other being observations and measurements from experiment.Similarly, physics education has a tradition of emphasizing theory and concepts in lecture and also providing experimental opportunities for students in lab courses.Our study investigated students' views about experiments and their relation to theory across a broad student population.
Halloun's framework on students' personal epistemology in science breaks into two dimensions (cognitive and scientific) [14].We adapted the three sub-categories within the scientific dimension into our study design: structure, methodology, and validity.We developed, administered, and analyzed results from an open-ended survey that was given to physics students ranging from college freshmen to Ph.D. students.

II. METHODOLOGY
The development of the survey involved several phases: first, we generated open-ended questions; second, we conducted individual interviews with eight students ranging from introductory physics to Ph.D. students and used interview data to shorten and clarify survey questions; third, we created an online version of the survey and tested it with three students to ensure that it could be completed within 15 minutes.In this article, we focus our discussions on the structure and methodology categories of students' personal epistemology which were probed by the first four questions, which are: (Q1) "In your opinion, why are experiments a common part of physics classes?Provide examples or any evidence to support your answer.";(Q2) "In your opinion, why do scientists do experiments for their research?Provide examples or any evidence to support your answer.";(Q3) "In your opinion, what defines a scientific theory?";(Q4) "How do theory and experiment relate?Provide examples or any evidence to support your answer."The analysis of other survey questions related to validity will be presented in a later publication.
The survey was administered to: 75 students from introductory algebra-based physics courses with labs aligned with lecture content, 28 students from an introductory calculus-based physics course in an integrated lecture/lab studio environment, 20 upper-division physics majors from two stand-alone modern physics lab classes, and 31 physics Ph.D. students.All undergraduate students were from a large private university and the survey was administered late in the semester.Ph.D. students were from a large public research university and the survey was taken at the beginning of summer term.Students took the survey outside of class within one to two weeks of receiving the survey link in their emails.The survey was meant to investigate students' views based on their overall experiences in physics, not just their current course or project.In introductory and upper-level courses, the instructors offered extra credit to students who completed the survey.Ph.D. students were provided with gift cards to compensate for their time.All responses were imported into NVivo qualitative data analysis software and the coding structure were emergent from the data.The codes were mainly developed by one coder and were refined upon the discussion of both coders.The final percentage agreement between the two coders for all codes was above 90%.

A. Code definitions
All emergent codes (shown in bold italics) from each question are defined below.
Q1 "Why are experiments a common part of physics classes?"Supplemental learning -Experiments provide supplemental learning experiences for concepts and theories.Theory testing -The purpose of doing physics experiments is to prove a theory or test a hypothesis.B. Finding: Experiments as supplemental learning experiences vs. the foundation of physics One of the most striking, though perhaps unsurprising, findings was that introductory students almost exclusively emphasized experiments as tools for supplemental learning of theories and concepts (italics denote a code defined in Sec.III A).In Q1, "Why are experiments a common part of physics classes?", about 91% of algebra-based and 100% of calculus-based student responses were coded as supplemental learning, which strongly contrasts with the low percentages (< 25%) in all other codes for Q1.Many introductory students provided illustrations for why experimentation could be a good learning tool for concepts.One algebra-based student said, "I believe that experiments help students visualize the theories that they are learning."Another said that experiments "...demonstrate to students that the concepts they're learning are present in the real world."Students routinely pointed to the hands-on and visual experiences of the lab as important modes for supplementing the lecture material.A calculus-based student said, " [Experiments] give students a hands on and visual example of real life physics problems...Since some people can be more visual learners, this can be very beneficial."For the upper-division and Ph.D. students, experiments were valued for unique things they provided in addition to supplemental learning of lecture material, including the role of experiments in the foundation of physics and the ways that experiments develop scientific abilities and other professionally relevant skills.74% of Ph.D. students emphasized the empirical nature of physics (coded as foundation of physics).One Ph.D. student said that "Physics experiments are the core of physics.Theory is nice and math can be beautiful, but it is the experimental evidence of an experiment that gives this field truth."An example of developing scientific abilities was "... experimental skills are often more important than other skills taught in a physics class, which are typically mathematical skills.The ability to take a measurement is just as important (if not more important) of a skill to have outside of the classroom."Some students pointed out that labs foster an appreciation of science.For example, "By including experiments in physics classes, students might see how other scientists have used experiments to help resolve questions or better understand a particular phenomena..." For these students, experimental experiences are intrinsically valuable, as opposed to being valued merely for improving performance on homework assignments or exams.
The emphasis on experiments as tools for improving content learning in lecture is unsurprising given that many faculty have similar goals for labs [15].However, it is somewhat surprising given the small amount of evidence that lab courses are helpful for learning concepts [16], and some studies suggest simulations may support conceptual learning better than hands-on experiments [17].What is more concerning is that introductory students largely missed what is uniquely teachable in a lab environment (e.g., engagement in scientific practices, and a broader view of the role of evidence in the scientific process).Doctoral students with years of training in physics seemed to gain a broader understanding of the value of experimentation.Our finding is also consistent with Lippmann's study on introductory students' perceptions of the goals of labs, that is to perceive conceptual learning as one major goal for lab courses [18].However, lab curricula with an explicit emphasis on cultivating scientific abilities and experimental skills have changed students' perceptions of lab courses [19].

C. Finding: Experiment tests theory vs. a cyclical relationship between theory and experiment
The second striking finding relates to the way students described theory and its relationship with experiment.Introductory students viewed experiments primarily as means to test or provide evidence for a theory, while upper-division and Ph.D. students additionally emphasized experiments inspiring new theories, and the ways in which theory explains and guides experiments.
Question 2 asked, "Why do scientists do experiments for their research?"All levels of students mentioned the importance of theory testing.An introductory algebrabased student said that experiments "test ideas under controlled circumstances, and see if they hold merit in the physical world rather than just the world of theory."In addition, the majority of Ph.D. students mentioned doing experimental research for discovery.
Question 3 asked "What defines a scientific theory?"The only major code from introductory algebra-based students was evidence supported, though the majority of higher-level students also recognized this feature.One example from an algebra-based student was "A scientific theory is backed by experimental proof, and is assumed to be true until proven otherwise."Explanatory and predictive power is the major code that distinguishes higher-level students from introductory students.One Ph.D. student said, "A scientific theory provides a theoretical description of a phenomena that is able to predict experimental results correctly..." Nature is full of interesting phenomena worthy of experimental investigation, and theories provide concise explanations of those results.Prior research has similarly shown that the awareness of the explanatory power of scientific theories was a distinguishing feature between expert and novice views on the nature of science [5].Upper-division and Ph.D. students also recognized the quantitative aspect of theories, which in physics are usually expressed as a mathematical construct involving symbols and equations.
Question 4 asked "How do theory and experiment relate?"Similar to their answers in Q2, 79% of algebrabased and 96% of calculus-based students recognized experiment tests theory as a key component.Most introductory students provided a brief response, such as, "Theory is an idea, experiment is putting the idea to the test," or "The experiments prove the theory."However, very few recognized other features.Upper-division and Ph.D. students identified a holistic cyclical relationship between theory and experiment, which included: observations from experiments inspire theory development (experiment inspires theory), a theory can then lead to the design of new experiments and testable predictions (theory guides experiment), theories are modified or developed to explain experimental findings (theory explains experiment), and the whole process is cyclical and iterative.One Ph.D. student said, "Theory and experiment are different and valuable ways to investigate nature.Sometimes progress in physics is made by an experimental discovery of some phenomena, which excites theorists who can then formulate a theory to describe it.An example of this is the Quantum Hall Effect.Other times, a theoretical prediction guides experimental efforts.E.g., the search and discovery of the Higgs boson."

IV. CONCLUSIONS
In this study, we identified differences between introductory and higher-level students in their personal epistemological views of physics with a specific focus on experiments.First of all, introductory students viewed experiments primarily as a way to supplement learning in lecture while higher-level students identified many unique benefits of doing experiments.Secondly, when reflecting on the roles of experiments, introductory students heavily emphasized experiments for testing theories, while higher-level students provided a more holistic picture of theory and the relationship between theory and experiment.It is possible that the heavy emphasis on conceptual learning and problem solving is shaping students' personal epistemology in unintended ways in physics courses by hiding or downplaying the diverse purposes of experiments in physics.In a more recent study from Colorado Learning Attitudes about Science Survey for Experimental Physics (E-CLASS), the findings from a large national data set indicated that students in skills-focused courses showed more expert-like responses and more favorable shifts than students in either concepts-focused or both-focused courses [15].
We took snapshots of students with varying physics lab experiences.This study gave us insights into the trajectory of students' epistemological progression in physics which was driven by our current undergraduate curriculum.We would like to develop a modified survey format that is easier to analyze across a larger population.Future work may include a longitudinal study tracking the same group of students over many years.
Experiment inspires theory -Experiments can inspire the development or modification of theories.Theory guides experiment -Theories often provide the basis for the design of experiments.The distributions of major codes across all four questions are shown in Fig. 1.
Foundation of physics -Physics concepts/laws are grounded in experiments/observations.Scientific abilities -Experiments help cultivate students' scientific abilities, such as experimental design, data collection and data analysis skills.Science appreciation -Doing experiments helps stu-dents understand how scientists study the world and make students appreciate science.Career preparation -Training in physics experiments is helpful for future career in physics and related fields.Q2 "Why do scientists do experiments for their research?"Theory testing was defined above under Q1.Experiment tests theory -Experiments are conducted to test, support or back up a theory.Theory explains experiments -Theories can explain or predict observations and experiment results.