Analyzing student understanding of vector field plots with respect to divergence

A survey with N = 39 second-year students demonstrates that most students fail to relate graphical vector field representations to the concept of divergence. Even after providing them with two visual strategies (integral vs. derivative strategy), students still struggle to indicate whether vector fields have zero or non-zero divergence. To gain further insight into student reasoning, we captured their visual attention on the field plot via eye tracking. Fixation patterns and relevant eye-tracking measures reveal that both visual strategies are cognitively processed differently, and that success of either strategy depends on field characteristics. Future research will be outlined.


I. INTRODUCTION
Divergence is a mathematical concept with many applications in physics.It appears, for example, in the differential form of Maxwell's equations ( ∇ • E = ρ 0 , ∇ • B = 0), or in continuity equations describing the transport of some quantity ( ∇ • J = − ∂ρ ∂t ).While these differential forms apply at a specific point, any continuity equation can also be expressed in an integral form (in terms of a flux integral) referring to a finite region, e.g.V ∇• JdV = ∂V J •d n (Gauss's theorem).
Both representations of divergence find applications in many scientific disciplines, e.g.engineering, oceanography, gravimetry, atmospheric sciences, etc. Therefore and because of the crucial role of representational competence for scientific expertise [1,2], it is important that students develop visual understanding of the mathematical concept to fully capture the above mentioned physical relationships.
However, prior research showed that both, graduate and undergraduate students, have conceptually difficulties with divergence [3][4][5][6].While being able to evaluate mathematical expressions, students have no coherent picture about the meaning of divergence [4], and struggle to correctly identify divergence given graphical representations of vector fields [5,6], such as those given in Fig. 1.Contributing to this line of research, we first introduce the visual divergence problem (VDP) as the starting point of our research (Section II).To gain insight into student visual processing of vector fields and the effectiveness of different visual strategies to solve the VDP, we conducted an instruction-based eye-tracking study under controlled conditions.After presenting basic theoretical background of eye tracking (Section III), we describe the study including research questions, pre-instruction survey, design, material, and eye-tracking procedures (Section IV).Last, we present results concerning student performance, eyetracking measures, and their correlations.We conclude with a brief discussion.

II. SOLVING THE VISUAL DIVERGENCE PROBLEM (VDP)
Given a graphical vector field representation (cf.Fig. 1), students decide whether divergence is zero or non-zero everywhere.This problem was inspired by prior work in this field [5], and we refer to it as the VDP.Please note that divergence of all vector fields used in our research is either zero or nonzero for each position in the field.Students were informed about this restriction because most of them (85%) knew that divergence is a local property of a vector field, as the pretest revealed.
A. Strategy 1: The differential approach Divergence of a vector field F can be evaluated using its differential representation in Cartesian coordinates, i.e., div Given a visual field representation, the right hand side of Eq. ( 1) instructs looking for changes of the field components over the xand y-direction (partial vector derivatives) starting in a certain point.Vividly speaking, the eye will perform horizontal and vertical gaze patterns above the vector field.
If one or more field-component is zero (e.g.F x = 0, F z = 0), this task becomes straightforward.Cases in which changes of the field components in the xand the ydirection cancel each other are not considered.

B. Strategy 2: The integral approach
Divergence can also be evaluated using its integral representation, i.e., ( Given a visual field representation, one can estimate divergence with the flux through the boundary ∂V of a test-volume V in the field.Test volumes could take any form, but its appropriate to make use of field's symmetry such that the outer surface normal d n is either parallel or perpendicular to the field vector F (e.g.rectangles in case of vector fields a or b, cf.Fig. 1).One can then balance positive and negative flux.

III. THEORETICAL BACKGROUND: EYE-TRACKING AND COGNITION
Research has shown that competent handling of representations is a prerequisite to develop scientific expertise, problem solving, and understanding of physical concepts [1,2].Therefore, it is important to discover how students read and use visual representations.Eye-Tracking is a non-intrusive method to obtain information about visual attention and cognitive processing while students solve problems, such as the VDP.A robust body of research has shown that eye-gaze may be considered an unbiased indicator of the focus of visual attention [7], and that eye-tracking measures can be related to performance measures, particularly in the context of proessing visual stimuli [8].We use this non-intrusive method to gain further insight into student cognitive processing of visual representations.
The most often used eye-tracking measures are derived from fixations (relatively long periods, usually lasting between 100 and 600 ms, in which the eye is almost still) and saccades (very fast eye shifts between fixations, lasting less than 100 ms): Fixation duration, number of fixations, and saccade length (cf.Fig. 2).According to the theory of longterm working memory, the information-reduction hypothesis, and the holistic model of image perception, these three measures are associated with information processing, selective attentional allocation, and visual span, respectively [8].Hence, these measures can give reliable indications of task difficulty (in term of required cognitive effort) and student expertise.

IV. INSTRUCTION-BASED EYE-TRACKING STUDY A. Research questions
In this contribution, we address the following research questions: 1. Which visual strategy promotes better learning, as revealed by performance scores? 2. Are both strategies cognitively processed differently?
In other words, do eye-tracking measures differ between both VDP-solving strategies? 3. Are there significant correlations between eye-tracking measures and measures of student performance?
The Purpose of research question 3 is to discover the potential of eye-tracking measures to discriminate successful and unsuccessful problem solving.

B. Sample
Thirty-nine major physics students of the University of Kaiserslautern (35 male) aged 19-24 (average 20.6 years) took part in the experiment.All participants were about to attend an introductory electromagnetism course (total enrollment N = 72) and had successfully completed two mechanics lectures (calculus-based mechanics and experimental physics).Divergence has been introduced in both mechanics lectures, and has also been recapitulated in the electromagnetism course before the experiment was conducted.Participation was voluntary, took 30 min in total (survey and experiment), and was compensated with 10$.From pre-survey results, we can conclude that students had little problems to identify xand y-components of vector fields but had some minor problems with the concept of flux through the boundary.Only very few students ( 3 out of 39) were able to solve the VDP prior to instruction.

D. Study design and material
After completing the pre-instruction survey, students started the instruction-based experiment displayed on a computer screen.The sequence of the experiment is illustrated in Fig. 3. Depending upon chance, each student either started with strategy 1 (see II A; the derivative strategy, DS) or strategy 2 (see II B; the integral strategy, IS).Both instructions, DS and IS, covered 250 words (1 textbook page), respectively, and included a step-by-step description with visual cues about application (worked-out example).In each instruction period, students applied the prevailing strategy to four vector fields (V1-V4) which were presented one after another.The vector fields used in the study are presented in Fig. 1.Students did not receive any feedback after completing a VDP, and were unable to skip back to the instruction page.Each student completed both instruction periods in order to balance group effects.Students were told to apply only the current strategy.

E. Eye-tracking procedures and measures
We obtained gaze data for all 39 students using a Tobii X3-120 eye-tracker installed on a 24" LCD screen as they worked with the VDP.All students had normal or correct-to-normal vision.The device has an accuracy of 0.4 degrees and allows a relatively high freedom of head movement.Its sampling frequency is 120 Hz.Gaze recording was accomplished using the Tobii Studio Pro software, able to detect saccades, fixations, and visualizations of data.From this data, we extracted the following measures for all students and every vector field 1-4 in each condition (DS, IS): • Number of fixations; that is the number of all fixations within the vector field plot of each VDP.
• Fixation duration; that is duration of each individual fixation within the field plot.
• Saccade length; that is distance of eye movements between two fixations.

V. RESULTS AND DISCUSSION
A. RQ1: Student performance vs. strategy Results are shown in Tab.I.The derivative strategy yielded a correct response in 62% of cases, and the integral strategy in 70% of cases.Keeping in mind that the chance guessing correct is 50%, we conclude that even after instruction students total performance on the VDP is mediocre.When only the first instruction period is considered, total performance score is 0.60.Considering the overall score of 0.66, we can conclude that students slightly improved their reasoning in the second instruction period, independent from the specific instruction.
From the results we obtain evidence that the effectiveness of each strategy depends upon the specific field (Cohen's d = 0.72-1.02).It is notable that the derivative strategy seems to fail when the vector field has no divergence (cases b and c) but is superior when divergence is non-zero (cases a and d).In other words, students are more inclined to accredit divergence to vector fields when using the derivative strategy.This may be due misinterpreting the partial vector derivative, cf.Eq. ( 1), and needs further investigation.II represent averaged statistics over all four vector-field items.When students apply the integral strategy, they made significantly more fixations on the visual representation as compared to using the derivative strategy.According to the information reduction hypothesis (cf.Section III) this means that students need to attend to more information leading to a longer processing time.This effect becomes even more pronounced (d = 1.0) when only the first instruction periods are being compared (Fig. 3).

Results shown in Table
Differences between fixation durations are also marginal longer when the integral strategy is applied supporting the hypothesis that processing requires more cognitive effort.Indeed, students need to construct a mental representation of a test volume in order to balance the flux through its boundary which may cause cognitive load.In contrast, the derivative strategy does not require to construct an additional mental representation in all cases.For example, there is no vector decomposition needed for the fields (a) and (b), cf.Fig. 1.We also found striking evidence for longer saccade lengths when students worked with the derivative strategy.According to the holistic model of image perception [8], students extract information from widely distanced and parafoveal regions.This occurs, e.g., when students explore the change of the field across horizontal or vertical directions.

C. RQ3: Correlations between eye-tracking measures and student performance
The analysis was restricted to the most difficult vector fields (in terms of student performance); task (c) in case of the DS, and task (a) in case of the IS, respectively (cf.Table I).The analysis refers only to the first instruction period in order to preclude learning effects when the vector field is processed visually.
Concerning task (c), students who answered correctly had significant larger saccade lengths (r = 0.59, p < 0.01) than students who answered incorrectly.This could be accredited to a appropriate application of the DS (scanning field vectors for changes) and experts extended visual span [8].However, this correlation was not existent when students applied the IS on task (b).This could be explained by the various possible test volumes students imagined when they solved the task.
The average saccade length might reflect processing mental images of small and large rectangles; thus this measure alone is no appropriate indication to distinguish correct and incorrect problem solving.In both cases, correct responses were weakly (anti)correlated with number of fixations (r = −0.26,p = 0.05), i.e. correct problem-solvers needed less fixations prior to responding.This finding supports the hypothesis that experts have shorter processing times [8], but it needs further investigation due to low statistical power (small sample size).There were no correlations between performance scores and fixation durations.

VI. CONCLUSIONS AND OUTLOOK
From the results, we obtain strong evidence that both problem-solving strategies complement each other: The effectiveness of either strategy fluctuates across specific VDPs (RQ1), and they promote different gaze behaviors (RQ2).Hence, both strategies are cognitively processed differently, and thus contribute to students visual flexibility which is considered a prerequisite of problem solving.We hypothesize that students should be taught both strategies to develop a coherent mental concept of divergence.This can be tested straightforward by adding a third group to our design which receives both strategies at the beginning.We further found correlations between performance scores and eye-tracking measures (saccade lengths, number of fixations; RQ3).Using just-in-time diagnostics, this can be useful to predict whether students need further instructional support (visual cues) giving great potential for future research.In a next step, the instructional strategies will be analyzed (reading behavior), manipulated (simulations), and linked to outcome measures.

FIG. 1 .
FIG. 1. Examples of vector field plots with zero (b, c) and non-zero divergence everywhere (a, d).All vector fields are two-dimensional and have no dependence on the z-coordinate.

FIG. 2 .
FIG.2.Gaze plot (left) and heatmap (right) for two different vector fields.Gaze plots contain information on individual level about the order of looking, saccade length and fixation duration.In contrast, heatmaps show how visual attention is distributed over the field representation for a subset of students.

C
FIG. 3. Study design

TABLE I .
Student performance scores on each VDP (cf.Fig.1).Statistics refer to a two-tailed t-test (DS vs. IS), N = 39.

TABLE II .
Comparison of eye-tracking measures (mean and standard deviation) between both strategies (cf.Sect.IV E).