Detail Page

Assessing the assessment: Mutual information between response choices and factor scores
written by Cole Walsh and N. G. Holmes
Validated formative assessment tools provide a reliable way to compare student learning across variables such as pedagogy and curricula, or demographics. Such assessments typically employ a closed-response format developed from student responses to open-response questions and interviews with students and experts. The validity and reliability of these assessments is usually evaluated using statistical tools such as classical test theory or item response theory. The suitability of individual questions on an assessment can be examined using either of these methods, but so far little attention has been given to evaluating the suitability of individual response choices available in each question. Here, we use mutual information, a tool rarely used in PER, to quantitatively evaluate the utility of response choices in an assessment. We use the Physics Lab Inventory of Critical thinking (PLIC) as an example to illustrate how assessment developers can use this method for their own assessments. The PLIC was designed to measure three latent constructs and we confirm this structure through a factor analysis. We calculate factor scores that represent performance on each of the proposed constructs and evaluate the suitability of individual response choices in terms of how much information they provide about these factor scores.
Physics Education Research Conference 2019
Part of the PER Conference series
Provo, UT: July 24-25, 2019
Pages 597-602
Subjects Levels Resource Types
Education Foundations
- Assessment
= Conceptual Assessment
= Instruments
- Research Design & Methodology
= Evaluation
= Statistics
= Validity
- Graduate/Professional
- Reference Material
= Research study
Intended Users Formats Ratings
- Researchers
- application/pdf
  • Currently 0.0/5

Want to rate this material?
Login here!


Mirror:
https://doi.org/10.1119/perc.2019…
Access Rights:
Free access
License:
This material is released under a Creative Commons Attribution 3.0 license. Further distribution of this work must maintain attribution to the published article's author(s), title, proceedings citation, and DOI.
Rights Holder:
American Association of Physics Teachers
DOI:
10.1119/perc.2019.pr.Walsh_C
NSF Number:
1611482
Keyword:
PERC 2019
Record Creator:
Metadata instance created December 31, 2019 by Lyle Barbato
Record Updated:
January 8, 2020 by Lyle Barbato
Last Update
when Cataloged:
December 31, 2019
Other Collections:

ComPADRE is beta testing Citation Styles!

Record Link
AIP Format
C. Walsh and N. Holmes, , presented at the Physics Education Research Conference 2019, Provo, UT, 2019, WWW Document, (https://www.compadre.org/Repository/document/ServeFile.cfm?ID=15222&DocID=5137).
AJP/PRST-PER
C. Walsh and N. Holmes, Assessing the assessment: Mutual information between response choices and factor scores, presented at the Physics Education Research Conference 2019, Provo, UT, 2019, <https://www.compadre.org/Repository/document/ServeFile.cfm?ID=15222&DocID=5137>.
APA Format
Walsh, C., & Holmes, N. (2019, July 24-25). Assessing the assessment: Mutual information between response choices and factor scores. Paper presented at Physics Education Research Conference 2019, Provo, UT. Retrieved April 27, 2024, from https://www.compadre.org/Repository/document/ServeFile.cfm?ID=15222&DocID=5137
Chicago Format
Walsh, Cole, and Natasha G. Holmes. "Assessing the assessment: Mutual information between response choices and factor scores." Paper presented at the Physics Education Research Conference 2019, Provo, UT, July 24-25, 2019. https://www.compadre.org/Repository/document/ServeFile.cfm?ID=15222&DocID=5137 (accessed 27 April 2024).
MLA Format
Walsh, Cole, and Natasha G. Holmes. "Assessing the assessment: Mutual information between response choices and factor scores." Physics Education Research Conference 2019. Provo, UT: 2019. 597-602 of PER Conference. 27 Apr. 2024 <https://www.compadre.org/Repository/document/ServeFile.cfm?ID=15222&DocID=5137>.
BibTeX Export Format
@inproceedings{ Author = "Cole Walsh and Natasha G. Holmes", Title = {Assessing the assessment: Mutual information between response choices and factor scores}, BookTitle = {Physics Education Research Conference 2019}, Pages = {597-602}, Address = {Provo, UT}, Series = {PER Conference}, Month = {July 24-25}, Year = {2019} }
Refer Export Format

%A Cole Walsh %A Natasha G. Holmes %T Assessing the assessment: Mutual information between response choices and factor scores %S PER Conference %D July 24-25 2019 %P 597-602 %C Provo, UT %U https://www.compadre.org/Repository/document/ServeFile.cfm?ID=15222&DocID=5137 %O Physics Education Research Conference 2019 %O July 24-25 %O application/pdf

EndNote Export Format

%0 Conference Proceedings %A Walsh, Cole %A Holmes, Natasha G. %D July 24-25 2019 %T Assessing the assessment: Mutual information between response choices and factor scores %B Physics Education Research Conference 2019 %C Provo, UT %P 597-602 %S PER Conference %8 July 24-25 %U https://www.compadre.org/Repository/document/ServeFile.cfm?ID=15222&DocID=5137


Disclaimer: ComPADRE offers citation styles as a guide only. We cannot offer interpretations about citations as this is an automated procedure. Please refer to the style manuals in the Citation Source Information area for clarifications.

Citation Source Information

The AIP Style presented is based on information from the AIP Style Manual.

The APA Style presented is based on information from APA Style.org: Electronic References.

The Chicago Style presented is based on information from Examples of Chicago-Style Documentation.

The MLA Style presented is based on information from the MLA FAQ.

Assessing the assessment: Mutual information between response choices and factor scores:


Know of another related resource? Login to relate this resource to it.
Save to my folders

Contribute

Related Materials

Similar Materials