Detail Page

Physical Review Physics Education Research
written by Joseph Wilson, Benjamin Pollard, John M. Aiken, Marcos D. Caballero, and H. J. Lewandowski
Surveys have long been used in physics education research to understand student reasoning and inform course improvements. However, to make analysis of large sets of responses practical, most surveys use a closed-response format with a small set of potential responses. Open-ended formats, such as written free response, can provide deeper insights into student thinking, but take much longer to analyze, especially with a large number of responses. Here, we explore natural language processing as a computational solution to this problem. We create a machine learning model that can take student responses from the Physics Measurement Questionnaire as input, and output a categorization of student reasoning based on different reasoning paradigms. Our model yields classifications with the same level of agreement as that between two humans categorizing the data, but can be done by a computer, and thus can be scaled for large datasets. In this work, we describe the algorithms and methodologies used to create, train, and test our natural language processing system. We also present the results of the analysis and discuss the utility of these approaches for analyzing open-response data in education research.
Physical Review Physics Education Research: Volume 18, Issue 1, Pages 010141
Subjects Levels Resource Types
Classical Mechanics
- General
Education Foundations
- Assessment
= Instruments
- Research Design & Methodology
= Data
= Validity
- Student Characteristics
= Skills
General Physics
- Measurement/Units
- Physics Education Research
- Lower Undergraduate
- Reference Material
= Research study
Intended Users Formats Ratings
- Researchers
- Professional/Practitioners
- Administrators
- Educators
- application/pdf
- text/html
  • Currently 0.0/5

Want to rate this material?
Login here!


Access Rights:
Free access
License:
This material is released under a Creative Commons Attribution 4.0 license.
Rights Holder:
American Physical Society
DOI:
10.1103/PhysRevPhysEducRes.18.010141
NSF Number:
PHY-1734006
Keywords:
PMQ assessment research, PMQ validity, Physics Measurement Questionnaire validity, measurement in physics
Record Creator:
Metadata instance created June 28, 2022 by Lyle Barbato
Record Updated:
July 17, 2023 by Caroline Hall
Last Update
when Cataloged:
June 2, 2022
Other Collections:

ComPADRE is beta testing Citation Styles!

Record Link
AIP Format
J. Wilson, B. Pollard, J. Aiken, M. Caballero, and H. Lewandowski, , Phys. Rev. Phys. Educ. Res. 18 (1), 010141 (2022), WWW Document, (https://doi.org/10.1103/PhysRevPhysEducRes.18.010141).
AJP/PRST-PER
J. Wilson, B. Pollard, J. Aiken, M. Caballero, and H. Lewandowski, Classification of open-ended responses to a research-based assessment using natural language processing, Phys. Rev. Phys. Educ. Res. 18 (1), 010141 (2022), <https://doi.org/10.1103/PhysRevPhysEducRes.18.010141>.
APA Format
Wilson, J., Pollard, B., Aiken, J., Caballero, M., & Lewandowski, H. (2022, June 2). Classification of open-ended responses to a research-based assessment using natural language processing. Phys. Rev. Phys. Educ. Res., 18(1), 010141. Retrieved May 3, 2025, from https://doi.org/10.1103/PhysRevPhysEducRes.18.010141
Chicago Format
Wilson, J, B. Pollard, J. Aiken, M. Caballero, and H. Lewandowski. "Classification of open-ended responses to a research-based assessment using natural language processing." Phys. Rev. Phys. Educ. Res. 18, no. 1, (June 2, 2022): 010141, https://doi.org/10.1103/PhysRevPhysEducRes.18.010141 (accessed 3 May 2025).
MLA Format
Wilson, Joseph, Benjamin Pollard, John Aiken, Marcos D. Caballero, and Heather J. Lewandowski. "Classification of open-ended responses to a research-based assessment using natural language processing." Phys. Rev. Phys. Educ. Res. 18.1 (2022): 010141. 3 May 2025 <https://doi.org/10.1103/PhysRevPhysEducRes.18.010141>.
BibTeX Export Format
@article{ Author = "Joseph Wilson and Benjamin Pollard and John Aiken and Marcos D. Caballero and Heather J. Lewandowski", Title = {Classification of open-ended responses to a research-based assessment using natural language processing}, Journal = {Phys. Rev. Phys. Educ. Res.}, Volume = {18}, Number = {1}, Pages = {010141}, Month = {June}, Year = {2022} }
Refer Export Format

%A Joseph Wilson %A Benjamin Pollard %A John Aiken %A Marcos D. Caballero %A Heather J. Lewandowski %T Classification of open-ended responses to a research-based assessment using natural language processing %J Phys. Rev. Phys. Educ. Res. %V 18 %N 1 %D June 2, 2022 %P 010141 %U https://doi.org/10.1103/PhysRevPhysEducRes.18.010141 %O application/pdf

EndNote Export Format

%0 Journal Article %A Wilson, Joseph %A Pollard, Benjamin %A Aiken, John %A Caballero, Marcos D. %A Lewandowski, Heather J. %D June 2, 2022 %T Classification of open-ended responses to a research-based assessment using natural language processing %J Phys. Rev. Phys. Educ. Res. %V 18 %N 1 %P 010141 %8 June 2, 2022 %U https://doi.org/10.1103/PhysRevPhysEducRes.18.010141


Disclaimer: ComPADRE offers citation styles as a guide only. We cannot offer interpretations about citations as this is an automated procedure. Please refer to the style manuals in the Citation Source Information area for clarifications.

Citation Source Information

The AIP Style presented is based on information from the AIP Style Manual.

The APA Style presented is based on information from APA Style.org: Electronic References.

The Chicago Style presented is based on information from Examples of Chicago-Style Documentation.

The MLA Style presented is based on information from the MLA FAQ.

Save to my folders

Contribute

Similar Materials