Detail Page

Physical Review Physics Education Research
written by Vincent P. Coletta and Jeffrey J. Steinert
Recently, Nissen et al. argued in this journal for the use of Cohen's d, in place of the more commonly used normalized gain, in the analysis of preinstruction and postinstruction scores on concept inventories used to measure the effectiveness of instruction. Their reason for advocating such a change is that they say normalized gains are "prescore biased." We provide five examples, including one cited by Nissen, that show no prescore bias when data are carefully analyzed, demonstrating that the problem with their analysis is omitted variable bias. We show that Cohen's d is less informative than normalized gain when used as a single parameter measure of teaching effectiveness, even though, as Nissen points out, d is more widely used in other fields. We believe that physics education researchers should continue to use normalized gain to assess educational effectiveness of pedagogy. However, because different student populations can have significantly different responses to the same pedagogy, in any interpretation of normalized gain, it is important to consider a measure of the abilities of the students. In analyzing normalized gains for the Force Concept Inventory (FCI), average scores on either Lawson's Test of Scientific Reasoning Ability or the SAT should be considered, because these scores are strongly correlated with normalized gain, indicating student abilities may have a greater impact on the gains achieved in a class than the specific pedagogy used.
Physical Review Physics Education Research: Volume 16, Issue 1, Pages 010108
Subjects Levels Resource Types
Education Foundations
- Assessment
= Conceptual Assessment
= Methods
- Research Design & Methodology
= Data
= Validity
- Sample Population
- Student Characteristics
= Ability
= Skills
General Physics
- Physics Education Research
- Scientific Reasoning
- Lower Undergraduate
- Reference Material
= Research study
Intended Users Formats Ratings
- Researchers
- Professional/Practitioners
- Administrators
- application/pdf
- text/html
  • Currently 0.0/5

Want to rate this material?
Login here!


Access Rights:
Free access
License:
This material is released under a Creative Commons Attribution 4.0 license.
Rights Holder:
American Physical Society
DOI:
10.1103/PhysRevPhysEducRes.16.010108
Keywords:
FCI validity, LST assessment, LST concept inventory, Lawson's Test
Record Creator:
Metadata instance created June 15, 2020 by Lyle Barbato
Record Updated:
January 20, 2023 by Caroline Hall
Last Update
when Cataloged:
February 6, 2020
Other Collections:

ComPADRE is beta testing Citation Styles!

Record Link
AIP Format
V. Coletta and J. Steinert, , Phys. Rev. Phys. Educ. Res. 16 (1), 010108 (2020), WWW Document, (https://doi.org/10.1103/PhysRevPhysEducRes.16.010108).
AJP/PRST-PER
V. Coletta and J. Steinert, Why normalized gain should continue to be used in analyzing preinstruction and postinstruction scores on concept inventories, Phys. Rev. Phys. Educ. Res. 16 (1), 010108 (2020), <https://doi.org/10.1103/PhysRevPhysEducRes.16.010108>.
APA Format
Coletta, V., & Steinert, J. (2020, February 6). Why normalized gain should continue to be used in analyzing preinstruction and postinstruction scores on concept inventories. Phys. Rev. Phys. Educ. Res., 16(1), 010108. Retrieved May 22, 2024, from https://doi.org/10.1103/PhysRevPhysEducRes.16.010108
Chicago Format
Coletta, Vincent, and Jeffrey J. Steinert. "Why normalized gain should continue to be used in analyzing preinstruction and postinstruction scores on concept inventories." Phys. Rev. Phys. Educ. Res. 16, no. 1, (February 6, 2020): 010108, https://doi.org/10.1103/PhysRevPhysEducRes.16.010108 (accessed 22 May 2024).
MLA Format
Coletta, Vincent, and Jeffrey J. Steinert. "Why normalized gain should continue to be used in analyzing preinstruction and postinstruction scores on concept inventories." Phys. Rev. Phys. Educ. Res. 16.1 (2020): 010108. 22 May 2024 <https://doi.org/10.1103/PhysRevPhysEducRes.16.010108>.
BibTeX Export Format
@article{ Author = "Vincent Coletta and Jeffrey J. Steinert", Title = {Why normalized gain should continue to be used in analyzing preinstruction and postinstruction scores on concept inventories}, Journal = {Phys. Rev. Phys. Educ. Res.}, Volume = {16}, Number = {1}, Pages = {010108}, Month = {February}, Year = {2020} }
Refer Export Format

%A Vincent Coletta %A Jeffrey J. Steinert %T Why normalized gain should continue to be used in analyzing preinstruction and postinstruction scores on concept inventories %J Phys. Rev. Phys. Educ. Res. %V 16 %N 1 %D February 6, 2020 %P 010108 %U https://doi.org/10.1103/PhysRevPhysEducRes.16.010108 %O application/pdf

EndNote Export Format

%0 Journal Article %A Coletta, Vincent %A Steinert, Jeffrey J. %D February 6, 2020 %T Why normalized gain should continue to be used in analyzing preinstruction and postinstruction scores on concept inventories %J Phys. Rev. Phys. Educ. Res. %V 16 %N 1 %P 010108 %8 February 6, 2020 %U https://doi.org/10.1103/PhysRevPhysEducRes.16.010108


Disclaimer: ComPADRE offers citation styles as a guide only. We cannot offer interpretations about citations as this is an automated procedure. Please refer to the style manuals in the Citation Source Information area for clarifications.

Citation Source Information

The AIP Style presented is based on information from the AIP Style Manual.

The APA Style presented is based on information from APA Style.org: Electronic References.

The Chicago Style presented is based on information from Examples of Chicago-Style Documentation.

The MLA Style presented is based on information from the MLA FAQ.

Save to my folders

Contribute

Similar Materials