|
Using an LLM to Investigate Students' Explanations on Conceptual Physics Questions
Analyzing students' written solutions to physics questions is a major area in PER. However, gauging student understanding in college courses is bottlenecked by large class sizes, which limits assessments to a multiple-choice (MC) format for ease of grading. Although sufficient in quantifying scientifically correct conceptions, MC assessments do not uncover students' deeper ways of understanding physics. Large language models (LLMs) offer a promising approach for assessing students' written responses at scale. Our study used an LLM, validated by human graders, to classify students' written explanations to three questions on the Energy and Momentum Conceptual Survey as correct or incorrect, and organized students' incorrect explanations into emergent categories. We found that the LLM (GPT-4o) can fairly assess students' explanations, comparable to human graders (0-3% discrepancy). Furthermore, the categories of incorrect explanations were different from corresponding MC distractors, allowing for different and deeper conceptions to become accessible to educators.
Physics Education Research Conference 2025
Part of the PER Conference series Washington, DC: August 6-7, 2025 Pages 399-404
ComPADRE is beta testing Citation Styles!
<a href="https://www.compadre.org/portal/items/detail.cfm?ID=17173">Savage, Sean, and N. Sanjay Rebello. "Using an LLM to Investigate Students' Explanations on Conceptual Physics Questions." Paper presented at the Physics Education Research Conference 2025, Washington, DC, August 6-7, 2025.</a>
S. Savage and N. Rebello, , presented at the Physics Education Research Conference 2025, Washington, DC, 2025, WWW Document, (https://www.compadre.org/Repository/document/ServeFile.cfm?ID=17173&DocID=6101).
S. Savage and N. Rebello, Using an LLM to Investigate Students' Explanations on Conceptual Physics Questions, presented at the Physics Education Research Conference 2025, Washington, DC, 2025, <https://www.compadre.org/Repository/document/ServeFile.cfm?ID=17173&DocID=6101>.
Savage, S., & Rebello, N. (2025, August 6-7). Using an LLM to Investigate Students' Explanations on Conceptual Physics Questions. Paper presented at Physics Education Research Conference 2025, Washington, DC. Retrieved November 10, 2025, from https://www.compadre.org/Repository/document/ServeFile.cfm?ID=17173&DocID=6101
Savage, Sean, and N. Sanjay Rebello. "Using an LLM to Investigate Students' Explanations on Conceptual Physics Questions." Paper presented at the Physics Education Research Conference 2025, Washington, DC, August 6-7, 2025. https://www.compadre.org/Repository/document/ServeFile.cfm?ID=17173&DocID=6101 (accessed 10 November 2025).
Savage, Sean, and N. Sanjay Rebello. "Using an LLM to Investigate Students' Explanations on Conceptual Physics Questions." Physics Education Research Conference 2025. Washington, DC: 2025. 399-404 of PER Conference. 10 Nov. 2025 <https://www.compadre.org/Repository/document/ServeFile.cfm?ID=17173&DocID=6101>.
@inproceedings{
Author = "Sean Savage and N. Sanjay Rebello",
Title = {Using an LLM to Investigate Students' Explanations on Conceptual Physics Questions},
BookTitle = {Physics Education Research Conference 2025},
Pages = {399-404},
Address = {Washington, DC},
Series = {PER Conference},
Month = {August 6-7},
Year = {2025}
}
%A Sean Savage %A N. Sanjay Rebello %T Using an LLM to Investigate Students' Explanations on Conceptual Physics Questions %S PER Conference %D August 6-7 2025 %P 399-404 %C Washington, DC %U https://www.compadre.org/Repository/document/ServeFile.cfm?ID=17173&DocID=6101 %O Physics Education Research Conference 2025 %O August 6-7 %O application/pdf %0 Conference Proceedings %A Savage, Sean %A Rebello, N. Sanjay %D August 6-7 2025 %T Using an LLM to Investigate Students' Explanations on Conceptual Physics Questions %B Physics Education Research Conference 2025 %C Washington, DC %P 399-404 %S PER Conference %8 August 6-7 %U https://www.compadre.org/Repository/document/ServeFile.cfm?ID=17173&DocID=6101 Disclaimer: ComPADRE offers citation styles as a guide only. We cannot offer interpretations about citations as this is an automated procedure. Please refer to the style manuals in the Citation Source Information area for clarifications.
Citation Source Information
The AIP Style presented is based on information from the AIP Style Manual. The APA Style presented is based on information from APA Style.org: Electronic References. The Chicago Style presented is based on information from Examples of Chicago-Style Documentation. The MLA Style presented is based on information from the MLA FAQ. Using an LLM to Investigate Students' Explanations on Conceptual Physics Questions:Know of another related resource? Login to relate this resource to it. |
ContributeRelated MaterialsSimilar Materials |