This record is currently in the approval process for inclusion into the Open Source Physics collection.
If you feel this record should be given priority you may contact the editor using our feedback form.
Assessing the assessment: Mutual information between response choices and factor scores
written by
Cole Walsh and N. G. Holmes
Validated formative assessment tools provide a reliable way to compare student learning across variables such as pedagogy and curricula, or demographics. Such assessments typically employ a closed-response format developed from student responses to open-response questions and interviews with students and experts. The validity and reliability of these assessments is usually evaluated using statistical tools such as classical test theory or item response theory. The suitability of individual questions on an assessment can be examined using either of these methods, but so far little attention has been given to evaluating the suitability of individual response choices available in each question. Here, we use mutual information, a tool rarely used in PER, to quantitatively evaluate the utility of response choices in an assessment. We use the Physics Lab Inventory of Critical thinking (PLIC) as an example to illustrate how assessment developers can use this method for their own assessments. The PLIC was designed to measure three latent constructs and we confirm this structure through a factor analysis. We calculate factor scores that represent performance on each of the proposed constructs and evaluate the suitability of individual response choices in terms of how much information they provide about these factor scores.
Physics Education Research Conference 2019
Part of the PER Conference series
Provo, UT: July 24-25, 2019
Pages 597-602
This material is released under a Creative Commons Attribution 3.0 license.
Further distribution of this work must maintain attribution to the published article's author(s), title, proceedings citation, and DOI.
<a href="https://www.compadre.org/OSP/items/detail.cfm?ID=15222">Walsh, Cole, and Natasha G. Holmes. "Assessing the assessment: Mutual information between response choices and factor scores." Paper presented at the Physics Education Research Conference 2019, Provo, UT, July 24-25, 2019.</a>
C. Walsh and N. Holmes, , presented at the Physics Education Research Conference 2019, Provo, UT, 2019, WWW Document, (https://www.compadre.org/Repository/document/ServeFile.cfm?ID=15222&DocID=5137).
C. Walsh and N. Holmes, Assessing the assessment: Mutual information between response choices and factor scores, presented at the Physics Education Research Conference 2019, Provo, UT, 2019, <https://www.compadre.org/Repository/document/ServeFile.cfm?ID=15222&DocID=5137>.
Walsh, C., & Holmes, N. (2019, July 24-25). Assessing the assessment: Mutual information between response choices and factor scores. Paper presented at Physics Education Research Conference 2019, Provo, UT. Retrieved November 12, 2024, from https://www.compadre.org/Repository/document/ServeFile.cfm?ID=15222&DocID=5137
Walsh, Cole, and Natasha G. Holmes. "Assessing the assessment: Mutual information between response choices and factor scores." Paper presented at the Physics Education Research Conference 2019, Provo, UT, July 24-25, 2019. https://www.compadre.org/Repository/document/ServeFile.cfm?ID=15222&DocID=5137 (accessed 12 November 2024).
Walsh, Cole, and Natasha G. Holmes. "Assessing the assessment: Mutual information between response choices and factor scores." Physics Education Research Conference 2019. Provo, UT: 2019. 597-602 of PER Conference. 12 Nov. 2024 <https://www.compadre.org/Repository/document/ServeFile.cfm?ID=15222&DocID=5137>.
@inproceedings{
Author = "Cole Walsh and Natasha G. Holmes",
Title = {Assessing the assessment: Mutual information between response choices and factor scores},
BookTitle = {Physics Education Research Conference 2019},
Pages = {597-602},
Address = {Provo, UT},
Series = {PER Conference},
Month = {July 24-25},
Year = {2019}
}
%A Cole Walsh %A Natasha G. Holmes %T Assessing the assessment: Mutual information between response choices and factor scores %S PER Conference %D July 24-25 2019 %P 597-602 %C Provo, UT %U https://www.compadre.org/Repository/document/ServeFile.cfm?ID=15222&DocID=5137 %O Physics Education Research Conference 2019 %O July 24-25 %O application/pdf
%0 Conference Proceedings %A Walsh, Cole %A Holmes, Natasha G. %D July 24-25 2019 %T Assessing the assessment: Mutual information between response choices and factor scores %B Physics Education Research Conference 2019 %C Provo, UT %P 597-602 %S PER Conference %8 July 24-25 %U https://www.compadre.org/Repository/document/ServeFile.cfm?ID=15222&DocID=5137
Disclaimer: ComPADRE offers citation styles as a guide only. We cannot offer interpretations about citations as this is an automated procedure. Please refer to the style manuals in the Citation Source Information area for clarifications.