Methodological issues in eliciting blend mental models : t he context of sound propagation

While constructing their understanding in various science areas, students go through transitional phases that may involve richly developed and consistently used mental models. These transitional models are unique and coherent cognitive structures composed of the elements of both scientifically accepted models and the commonly used initial alternative models. Such models have been previously referred to as hybrid models or blend models. We discuss the nature of model blending in the context of sound propagation and suggest several solutions to methodological issues that surface in their qualitative and quantitative eliciting, such as linking answer choices across multiple questions, which we called Linked Item Model Analysis, LIMA.


I. INTRODUCTION
Students commonly reach a target scientific understanding of concepts only after undergoing transitional knowledge stages in which their spontaneous, initially held ideas and scientifically accepted knowledge are either coexistent [1,2] or inherently intertwined i.e. blended [3,4].Fauconnier and Turner [5] call the later process cognitive blending and they laid theoretical foundations for its understanding and further refinement [6].In conceptual blending, two or more mental input spaces (small conceptual packets) combine to produce a blend space [5].The need for the notion of "hybrid knowledge" [7], "hybrid" mental models [3], "synthetic" models [8], and "blended" mental models [4,9] -has emerged organically in physics education research over the past two decades in association with a wide range of physics topics (see [10] and references therein).A mental model is an internal representation that serves as a structural analogue for explaining situations and physical processes.We focus in this paper on the blending of mental models, and issues in eliciting mental models, especially in the context of multiple choice survey.

FIG 1.
Blend models and blend model states as proposed by the authors [3,10] A blend mental model is a composite model that systematically combines different features of, typically two parental models [8,10], which are usually the common initial model and the target scientific model.To clarify terminology as we use it, parental models in this case are blended models and the result is a blend model.Fig. 1. describes their relationship in terms of smaller knowledge elements which may or may not be associated exclusively with one of the parental models.Consistency of the model use defines mental model states [2] as (i) pure (i.e. in which a single model is used across different contexts) or (ii) mixed in which students interchangeably use different models.We add to this classification (iii) no model state (insufficiently coherent cognitive space in which relevant elements are not tied together) and (iv) hybrid or blend model state in which students consistently use a single, blend model which is a composite of elements normally associated with exclusively one of the parental models [10].

II. GOALS AND RESEARCH QUESTIONS
Our goals were to create a (i) blend-model sensitive (ii) multiple choice assessment of mental models of sound propagation, that is (iii) capable of productively guiding instruction in real time.The research question on which we focus here is how to optimally circumvent the ambiguities associated with inherent overlaps in a network of parental and blend models.We address this question in the context of the novel analytical approach (Linked Item Model Analysis or LIMA), the developed test and the associated results.
Earlier research distinguished four generic models of sound propagation [10]: (i) Wave Model (scientifically accepted model); (ii) Independent Entity Model (common initial alternative model) in which the sound is a selfstanding entity different from the medium through which it propagates and it can propagate in vacuum); (iii) Intrinsic Model (blend) according to which particles of the medium (a wave model feature) move translationally away from the source (independent entity feature), either with or without additional vibration (a Wave Model feature); (iv) and Dependent Entity Model (blend) in which sound requires a material medium in order to propagate (Wave Model feature), but sound is a self-standing unit different from the medium (as in the Independent Model).
Summarized commonalities between the models is that according to the Wave and Intrinsic Models, sound is a specific motion of particles of the medium caused by the source of sound.And according to both Entity Models (Independent and Dependent), sound is a self-standing entity different from the medium through which it propagates.
In addition to different models of sound propagation, there is a specific understanding of what sound is, which may be associated with different mechanisms of propagation, thus adding to the complexity of blending.This understanding that sound is what we hear, i.e. it is exclusively what we hear, we refer to as "the Ear-born Model."This model is distinct from the other four in that by itself does not describe a mechanism of sound propagation.

III. METHODOLOGY AND METHODOLOGICAL CONCERNS
A generic difficulty with eliciting novice learners' mental models is that these models are frequently unstable.These models may also be incomplete and can contain contradictory elements [11].We understand the process of knowledge construction as a dynamic internal process which is largely affected and/or facilitated by external inputs [12] and social interactions.Therefore, the process of mental model (re)construction can be affected by the probing instrument itself, making the model a moving target.

A. Issues in eliciting blend models
Mapping blend mental models onto multiple choice answers have additional inherent complications compared to eliciting target and common initial models.Four key difficulties are: (i) Additional complexity.Mental models cannot be always viably reduced to simple statements that fit a single answer choice.And blend models can be considerably more complex than either initial or target model.
(ii) Internal variability: Because a model can have submodels, more than one answer choice in a question may correspond to the same model.For example, the Independent Entity model may be associated with variety of different motions (including a lack of motion) of particles of the medium, (iii) Answer degeneration: More than one model may be associated with the same answer choice.As an example, longitudinal movement of medium particles during sound propagation is consistent with the Wave, Independent Entity, Dependent Entity and Ear-Born Models.
(iv) Cross-model commonalities: Blend Models may cause considerable overlaps between multiple-choice answers to the same question so that in some cases different choices pertaining to the same question may have substantial commonalities.For example, according to both the Independent and Dependent Entity Models, if sound is created in the medium it passes through empty spaces in between the particles of the medium.On top of these difficulties, "correct" cognitive structures embedded together with incorrect entities within the blend models can easily result in false positive answers to a variety of standard questions which do not take blends into consideration.
We addressed the aforementioned issues while developing an inventory of mental models for sound propagation by iteratively combining data from student interviews with open-ended, semi structured multiple-choice questionnaires, by performing quantitative comparisons of answer choices obtained in large-scale surveys, and using several modes of expert validation including (mental model) role-playing.Initial stages of testing were conducted at a large public U.S. Midwestern university.By the end of the study, a diverse set of more than 2400 students (at all three educational levels and all three mathematical levels of tertiary introductory physics) enrolled in 13 different educational institutions in the USA and Croatia participated in the study.Details of this process have been reported elsewhere.[10,13] Below we concentrate on discussing key solutions that we deployed in mitigating issues associated with eliciting blend mental models.

IV. FINDINGS AND DISCUSSION
Iterative testing defined optimal contextual situations for eliciting models of sound propagation as (i) propagation in the air and (ii) propagation through a barrier (wall) -each in combination with the "propagation" through the vacuum.

A. Linked Item Model Analysis (LIMA)
Based on the discussion above, the key difficulty with eliciting blend models was that it was not possible to map different models on to answer choices on test items so that each choice would correspond to a single and unique model.Complexity of the targeted sound propagation mechanisms span combinations of answer choices across multiple questions.For this reason we could not use the Model Analysis method as described by Bao [2].Instead, we developed and utilized an approach to multiple choice testing in which combinations of answers to more than one question define the student's model(s), and model state.Combining multiple responses enabled us to not exceed five multiple choice (MC) options for each question and also to keep those choices sufficiently simple and thus comprehensible.
Mental models of sound propagation described in the introduction differ according to the answers they result in with respect to the following questions (a) What is sound?(b) What happens (to sound) without a medium?(c) What are the dynamics of the particles of a medium during the sound propagation?(d) How are these dynamics related to sound propagation?These distinctions are the foundations for the chosen test questions.Below we have paraphrased the test questions for both contexts: Q1.What is the basic mechanism of sound propagation in air/ a wall?Q2.How do particles of the medium vibrate, if at all, while the sound propagates?Q3.How do particles of the medium travel, if at all, while the sound propagates?Q4.What does this motion have to do with sound propagation -cause and effect relationship?Q5.What does this motion have to do with sound propagation -time relationship?Q6.What happens with sound propagation in the vacuum?
Two tests have been created, each pertaining to one contextual set: one for propagation through air (and vacuum) and one for propagation through a wall (and vacuum).
In order to allow for a variety of movements of the particles of the medium (associated with the sound) that students expressed in open-ended questions, two separate questions (Q2 & Q3) were needed to elicit the movements of the particles of the medium associated with the sound propagation.One more question was necessary to associate this motion to sound propagation.In the air context, a minimum of three questions were needed to determine the model.Additional test questions were used to determine students' consistency, i.e. their model state.In the wall context, which is more complex than the air context because of the larger number of the factors (i.e.components) involved, four and sometimes all six test questions were used to determine the model associated with a particular answer choice [13].A model is not necessarily ascribed to any question triplet and student may be classified into the "other" category associated with an unidentifiable model.This approach to model analysis in which a students' mental model is determined by answers given to multiple test questions we called Linked Item Model Analysis (LIMA).In LIMA, the answer set is analyzed as a whole to first determine if the student is self-consistent, i.e. if s/he uses a single model throughout the test.As explained above, students in a blend model state are not internally inconsistent (and therefore they are in a pure model state) although their answers to standard questions may appear that way.
If no match is found, the student is either in a mixed model state (inconsistent) or in a no model state.In that case, the analysis program identifies the different models that the student uses by combining answers from questions Q2 and Q3 (dynamics defining questions) with each of the remaining four questions respectively.Questions Q2 and Q3 are themselves a paired subset as they together determine how particles of the medium move while the sound propagates through it.The remaining question in each of these triplets determines the student's rationale for this movement.This way each test (air and wall context test) probes a student's model four times (through these four sets of question-triplets. The LIMA testing and analysis approach makes it possible to address the issues regarding blend models mentioned above and to elicit students' mental models (including blend models) and their model states (including blend model state).

B. Display of results in terms of mental models
The Excel ® -based analysis program displays results in the five different graphs that show: (1) Percentages of times that a particular model is used by a class as a whole (2) Percentages of students using a particular model at least once, (3) Movements of particles of the medium, (4) Students' model and ( 5) Correctness of the answers.Because of the space restrictions, we show here a sample graph which displays students' model states (Fig. 2).The display of the pure model states differentiates the correct model from other consistently used models.Similarly, the mixed states are differentiated between a few characteristic combinations.Another specific feature of LIMA is that each possible answer combination does not necessarily correspond to a model.At the university level we found that the average number of such combinations (not identified with a model and classified as "other") in all the tests taken during the large survey phase was 5.21% (Ntests=1281).This percentage was higher at the high school level (9.78%, Ntests=300) and at the middle school level (8.55%; Ntests=132).So it was less than 10% at any educational level.

C. Sound Model Inventory
The result of this study is a model inventory that elicits mental models of sound propagation either as a formative assessment (in real-time, during the instruction) or as a summative assessment.We propose here the name of the test as "Mental Model Assessment of Sound Propagation" or for short "MMASP."The test and associated analytical software, together with the analysis [13] detailing the test's validity and reliability parameters can be downloaded from: http://web.phys.ksu.edu/role/sound.The test can be used as a summative assessment as well.However, if each question is graded separately, this may result in an overestimate of students understanding.This can be avoided by taking into account the number of correct model-defining question triplets as a foundation for grading.The developed Excelbased analysis programs can be used to automate this process.

D. Another look into model distribution
As a brief and large-scale overview of the results we obtained with the test, Fig. 3 compares post-instruction results at three educational levels as obtained in the air context.As expected, college students (N=689) perform better than high school students (N=156) and high school students perform better than middle school students (N=64).However, the differences between those educational levels are not as large as one might expect.The figure shows a steady rise toward higher educational levels when the Wave and Intrinsic Models are compared.The Ear-born Model and (to a lesser extent) the Dependent Entity Model have the opposite trend.Both Entity Models remain strong after instruction at all three levels.

FIG 3.
Post instruction results at primary, secondary and tertiary levels as obtained by the air context of the survey.
Elsewhere we have discussed differences between preand post-instruction test results, large scale findings, implications for teaching and descriptive pictorial representations of the basic models [13].
In addition to limitations of other MC inventories in general, the limitations specific to this test as blend model eliciting tool are that the test may identify a "no model" state as a mixed model state and it can possibly identify a pure model state if a student picks a model consistently.

A. Mental Model Assessment of Sound Propagation (MMASP)
We developed a mental model inventory for sound propagation named "Mental Model Assessment of Sound Propagation" or for short "MMASP."The inventory can be used as a formative assessment during instruction or as a summative test with certain restrictions associated with the answer analysis as the test is designed to take into account answers to several questions simultaneously.

B. Linked Item Model Analysis (LIMA)
Four basic generic models of sound propagation identified in earlier studies cannot be feasibly elicited through multiple-choice questionnaires in a way that a single model is mapped on a single answer choice (at least not with any reasonable number of offered choices).Instead, we combined and thus linked the answer choices in several questions for this purpose.We called this method of model eliciting LIMA (Linked Item Model Analysis).
LIMA approach can especially benefit model elicitation in many of the physics domains where there is evidence of blended models (see [10] and references therein).Suggestions for further research on this topic include (i) investigating those opportunities and (ii) optimizing the result display to facilitate informed teaching in real time.University, CC (11,1)

FIG 2 .
FIG 2. Display of students' model states in the test results.