Attention differences in viewing physics diagrams among experts and novices

It is known that experts identify/perceive meaningful patterns in visual stimuli related to their domain of expertise. This study explores the speed with which experts and novices detect changes in physics diagrams. Since change detection depends on where individuals direct their attention, differences in the speed with which experts and novices detect changes to diagrams would suggest differences in attention allocation between experts and novices. We present data from an experiment using the "flicker technique,” in which both physics experts and physics novices viewed nearly identical pairs of diagrams that are representative of typical introductory physics situations. The two diagrams in each pair contain a subtle difference that either does, or does not change the underlying physics depicted in the diagram. Findings indicate that experts are faster at detecting physics-relevant changes than physics-irrelevant changes, however there is no difference in response time for novices, suggesting that expertise guides attention for experts when inspecting physics diagrams. We discuss the cognitive implications of our findings. PACS: 01.40.Fk, 01.40.Ha, 01.40.-d


I. INTRODUCTION
The nature of expertise has received much attention in cognitive science research.Several studies have investigated the ways in which experts encode and access their conceptual understanding in order to solve problems across different domains (see [1,2] for reviews).For example, experts in physics possess conceptual schemas consisting of principles, contexts, and constraints for applying them that can be retrieved in order to aid in problem solving [3].An interesting and under researched question is how content expertise affects the allocation of attention when viewing diagrams encountered when studying or problem solving.
Change blindness is a phenomenon where changes in visual scenes go unnoticed.This suggests that we may not attend to every feature available to us in a visual scene.For example, Simons and Levin [4] conducted a study in which participants were asked for directions by an experimenter.While they were giving directions, the participants were distracted and a confederate exchanged places with the experimenter.Only about half of the participants noticed a change in the person to whom they were giving directions.This suggests that individuals tend to only allocate attention to features of the environment which are meaningful to the schema activated by the context in which they are situated.
Recently this phenomenon has begun to be employed to study expertise in physics.Feil and Mestre [5] showed physics situations to both experts and novices and asked them to explain the solution to a qualitative question.While the participant was distracted, the diagram was changed so that the physics of the problem was either affected or not affected by the change.They found that experts, but not novices noticed the change, and only when the substituted diagram modified the physics in a significant way, unless the novice exhibited a deep understanding of the physics underlying the initial situation.
This study extends the findings from the previous study by investigating experts' and novices' ability to notice changes in diagrams using another experimental technique called the "flicker technique."In the flicker technique [6], a participant sitting in front of a computer display is asked to detect a change in a pair of images that quickly alternates on the screen.The original image appears on the screen for a brief time, followed by a black screen lasting for a shorter duration, followed by the changed version of the original image, followed by the black screen, followed by the original image, and so on.
Besides the obvious superficial differences between the two techniques, there are additional important differences between the change-event paradigm used in [5] and the flicker paradigm used in this study.In [5], none of the participants knew that the original diagram which they first studied was going to change before they began explaining it, whereas in this study the participants were expecting a change and were told that their job was to identify that change as quickly as possible.Thus, in [5], the encoding of the diagram and its underlying physics was untimed and deliberate, whereas in this study speed in finding the change is of paramount importance.In other words, while the previous study investigated the effect of knowledge on memory, this study looks at the effect of physics knowledge on the allocation of attention.The focus on the rate at which an anticipated change is detected allows us to investigate how experts and novices allocate attention when examining physics diagrams [7].
Research in visual cognition suggests that attention is needed to notice a change, and that changes that are integral to interpreting the meaning of a scene are detected faster compared to less integral changes [7][8][9].If expertise in physics helps direct attention to those regions in a physics diagram that display important information, then it is reasonable to expect that the speed of noticing physics-relevant changes to diagrams tracks with expertise.Indeed, two flicker studies using non-physics topics suggest that expertise or familiarity can help detect relevant changes in scenes faster.One study [10] demonstrated that people who were social users of alcohol or cannabis detected changes faster in photographs compared to non-users when those changes were alcohol-and cannabis-related.Another study [11] investigated the ability of experts and novices in American football to detect changes in photographs of a game and non-football related scenes (photos of traffic scenes); experts were players, coaches, or referees in an American football league in Germany while novices were students from the University of Göttingen who had little to no experience with the game.Findings revealed that the American football experts were faster at detecting changes that were meaningful to the game's strategy compared to game-irrelevant changes (e.g., a change whereby a football appeared on the ground versus a change in the shadow cast by a player) or to non-football related scenes.
Two hypotheses were tested in this study.First, we hypothesized that physics experts will notice physics-relevant (consequential) changes to diagrams faster than physicsirrelevant (inconsequential) changes, whereas beginning physics students and total novices will not.Second, we hypothesized that time differences between detecting consequential changes compared to inconsequential changes attenuates with diminishing expertise; that is, the time differences will be largest for experts, smallest for total novices, and in-between for undergraduates having finished an introductory course.
If the hypothesized differences are found, then the flicker technique is another useful technique to measure expertnovice differences.Measuring differences in reaction times for noticing changes in physics diagrams are likely due to experts directing their attention differently than novices.In other words, experts likely direct their attention preferentially to regions in diagrams that portray significant physics resulting in faster times to notice physics-relevant changes compared to regions that contain physics-irrelevant changes.Thus, this study will explore whether knowledge of physics guides attention differentially for experts and novices when they inspect physics diagrams having physics-relevant changes or physics-irrelevant changes.To date, no such study has been conducted in an academic subject like physics which requires highly specialized abstract knowledge.

II. EXPERIMENTAL DESIGN
To test the effect that physics knowledge has on response time three groups of participants were recruited; education students, physics novices, and physics experts.The education participants in the study consisted of 28 undergraduate students enrolled in an Educational Psychology course at the University of Illinois.Of these students seven had recent college physics experience and were not included in the data analysis.In addition, one student was excluded due to failing to follow the correct procedure resulting in inflated response times.The physics novices in the study consisted of 20 undergraduate students enrolled in an introductory calculusbased mechanics course at the University of Illinois.The physics experts consisted of 19 graduate student TAs who taught discussion sections in the introductory calculus-based mechanics course.
Our experiment utilized the flicker paradigm, in which one physics diagram (original) and a slightly changed version (changed) are alternated on the screen with a blank, black screen as a separator between each image.Two different types of diagram pairs are used.In one type, which will be referred to as physics-irrelevant (see Fig. 1) change pairs, the changed diagram has an inconsequential change compared to the original in terms of the physics.In the other type, which will be referred to as physics-relevant pairs (see Fig. 1), there is a consequential change in the physics of the situation depicted in the changed diagram compared to the original.We  Participants were randomly assigned to complete one of the four scripts.The experiment was administered to the novice group after they had had the requisite physics in the course related to the diagrams in the experiment.

III. ANALYSIS AND RESULTS
The coordinates of the second click were used to exclude data when participants incorrectly identified the changing region.The change in the diagrams was not noticed, as indicated by the participants clicking on a part of the diagrams that did not change, on 71 of the 1770 trials included in the analysis (4%).Only response time data from correct responses were included in the analyses since error responses may be very different from correct responses.Upon inspection, the distribution of response times was not normally distributed (skew = 3.30, kurtosis = 16.03).To address this we transformed the data by taking the natural logarithm of each response time [12].Following this transformation the distribution of the ln(response time) was more normally distributed (skew = .78,kurtosis = .30).Table 1 shows the average ln(response time) for the correct responses of both novices and experts.In order to examine the influence of expertise and physicsrelevant changes (Physics) on response time, a 3 x 2 (Expertise x Physics) mixed ANOVA with physics relevance as the repeated measure was conducted.Both expertise (F(2,56) = 4.61, p = .01)and physics relevance (F(1,56) = 4.76, p = .03)were significant.In addition, the interaction between physics relevance and expertise was significant (F(2,56) = 5.10, p = .01).The significant interaction indicates that these groups tend to exhibit different patterns of reaction times when identifying physics-relevant changes compared to physics-irrelevant changes.The interaction can be seen in Fig. 2. The same pattern of response times was found using the untransformed response times indicating that the effect is not a function of the log transformation.
To investigate the effect that expertise has on response time in more detail, dependent samples t-tests were conducted for each group.The results of these tests indicated that there was a significant difference in identifying physics-relevant changes compared to physics-irrelevant changes for physics experts (t(18) = 3.75, p = .002),but not for physics novices (t(19) = 1.35, p = .19),or education students (t(19) = -1.42,p = .17).

FIG 2. Mean response time for participants
This indicates that experts noticed physics-relevant changes faster than physics-irrelevant changes, whereas physics novices and education students did not.Education students, in fact, displayed the opposite pattern, noticing physics-irrelevant changes faster than physics-relevant changes, though not significantly.For physics experts this corresponds to a difference of almost half a second (479 ms), which represents a large effect (d = .95).Physics novices noticed physicsrelevant changes only slightly faster than physics-irrelevant changes, corresponding to a difference of 258 ms.For education students this corresponds to a difference of -342 ms.

FIG 3. Mean non-detection rate for participants
A similar pattern was identified in examining the error rates exhibited by each group.Physics experts displayed a lower non-detection rate on diagrams with physics-relevant changes (1.75%) than on diagrams with physics-irrelevant changes (3.86%).Both physics novices and education students displayed a similar non-detection rate on diagrams with physics-relevant changes (4.00% and 4.67% respectively) than on diagrams with physics-irrelevant changes (4.67% and 5.00% respectively).This can be seen in Fig. 3.

IV. DISCUSSION
This study explored two related hypotheses, namely that experts' knowledge guides their attention so that they will detect consequential (physics-relevant) changes to physics diagrams faster than inconsequential (physics-irrelevant) changes, and that this advantage diminishes with diminishing expertise.To do so we employed the flicker technique to measure speed in detecting two different types of changes to physics diagrams, namely those that were consequential to the underlying physics and those that were not.
Our findings support the view that high knowledge individuals do detect consequential changes to physics diagrams faster and more accurately than inconsequential changes, suggesting that their domain knowledge helps guide their attention to relevant portions of physics diagrams.Only the physics experts showed this advantage.This finding suggests that expectations about important features of the diagram seem to be influenced by content expertise and direct the individual's attention when viewing diagrams depicting a physics situation.
Further, our findings support the hypothesis that there is a progression in attention allocation as expertise develops.The education students in our study who had not had any recent physics experience, including no college physics experience, not only did not show a speed advantage in detecting consequential changes in the diagrams, but their pattern was opposite of that of the experts'-they noticed inconsequential physics changes to the diagrams faster than consequential changes, although this difference was not significant.The physics novice's pattern was closer to that of the expert group than the education group as predicted, although their faster detection of physics-consequential changes was also not significant.Most importantly, there was a clear change in attention towards noticing physics-relevant changes faster than physics-irrelevant changes from experts to novices, to education students, as predicted.
The lack of any significant advantage of physics-relevant information for education students suggests that these students may lack a guiding schema of physics knowledge to direct their attention during the search for changes.This has implications for introductory physics instruction: It may be that for those students who do not do well in physics courses, their difficulty in learning physics is compounded by an inability to extract the appropriate physics from pictorial and other representations used in teaching physics.This implication is also supported by the Feil and Mestre [9] study where ability to answer correctly a question administered to assess understanding of a physics situation was highly correlated with ability to notice a consequential physics change to a diagram in that change-blindness study.Future investigations of how ability to extract important information from physics representations relates to other skills (e.g., problem solving, problem categorization) could prove fruitful and might suggest ways of improving introductory physics instruction.
The present study applies the flicker paradigm from visual cognition to the study of expertise in physics.Our findings suggest that the flicker technique could be a potentially useful tool for studying and measuring expertise.As we acquire knowledge in a complex domain such as physics, we are better able to direct our attention judiciously and "see" more rapidly relevant features in pictorial and symbolic depictions of situations.
While this study did not focus on educational applications, the results presented above have potential educational implications which should be investigated further.The education students took longer to detect changes in both the physics-relevant and the physics-irrelevant conditions.This seems to indicate that complete novices may possess underdeveloped orienting schemas which allow them to extract information from diagrams.As such, the purpose of the task is important to consider when presenting and utilizing diagrams with novice students.Highlighting relevant diagram features during instruction, accompanied by explanations for why these diagram areas are relevant may be a productive pedagogical strategy.During assessment, relevant features of diagrams should be highlighted, unless the extraction of physics-relevant information from a diagram is one of the intended learning outcomes to be measured by the question.
measured participants' response time to detect the change in each diagram pair.Thirty pairs of diagrams in the style used in typical physics exercises were created for this study.The two diagrams in each pair contain a subtle difference.Fifteen of the diagram pairs contained physics-relevant changes (R1 -R15), and the remaining fifteen pairs contained physicsirrelevant changes (I1 -I15).

FIG 1 .
FIG 1. Example Irrelevant and Relevant Diagram Pairs Participants were shown the diagram pairs on a computer screen.The diagram pairs were alternated on a computer screen with a black screen inserted between each diagram slide; the diagrams were displayed for 200 ms, while the black screen appeared for 100 ms.Participants were given the instructions that their task was to identify the change between the two diagrams in each pair as it flickered on the screen.Participants pressed the space bar as soon as they noticed the change between the two diagrams, which stopped the flickering.They then used the cursor to select the region on the diagram that contained the change.After this second click, the program automatically advanced to the next diagram pair.Software recorded the response time (time to identify the

Table 1 .
Means and SD of Ln(Response Time)