Detail Page

Physical Review Physics Education Research
written by Jun-ichiro Yasuda, Michael M. Hull, and Naohiro Mae
This paper presents improvements made to a computerized adaptive testing (CAT)-based version of the FCI (FCI-CAT) in regards to test security and test efficiency. First, we will discuss measures to enhance test security by controlling for item overexposure, decreasing the risk that respondents may (i) memorize the content of a pretest for use on the post-test or (ii) share information about the items with their classmates who take the assessment later. Second, we will discuss measures to enhance test efficiency, so that a shorter test length can yield a desired accuracy and precision of the measurement. Specifically, we utilized collateral information in the form of a pretest proficiency estimate of each respondent for selecting items and estimating respondent proficiency level in the post-test. To shorten the total testing time further, we also allowed the test lengths to be different for the pre- and post-test. To analyze how these improvements affect the accuracy and precision (which we measure in terms of root-mean-square error) of Cohen's d, we conducted a Monte Carlo simulation and a post hoc simulation. Then, we calculated the minimal test length of the FCI-CAT whose accuracy and precision are equivalent to that of the paper-and-pencil version of the FCI. Consequently, we obtained the following three findings: (i) By using collateral information, we can achieve the accuracy and precision of the full-length FCI with fewer items via the FCI-CAT. (ii) For a class size of 40, we can control for test security while still reducing the sum of the pre- and post-test lengths of the FCI-CAT to a total of 33 items (17 items on the pretest and 16 items on the post-test), thereby reducing the testing time to 55%. (iii) If one's goal is to maximize test efficiency, the pretest length should be slightly larger than the post-test length. If the goal is to maximize test security, the pretest length should be smaller and the post-test length should be larger.
Physical Review Physics Education Research: Volume 18, Issue 1, Pages 010112
Subjects Levels Resource Types
Classical Mechanics
- General
Education Foundations
- Achievement
- Assessment
= Conceptual Assessment
- Research Design & Methodology
= Data
= Validity
General Physics
- Physics Education Research
- Lower Undergraduate
- Reference Material
= Research study
Intended Users Formats Ratings
- Administrators
- Professional/Practitioners
- Researchers
- Educators
- application/pdf
- text/html
  • Currently 0.0/5

Want to rate this material?
Login here!


Access Rights:
Free access
License:
This material is released under a Creative Commons Attribution 4.0 license.
Rights Holder:
American Physical Society
DOI:
10.1103/PhysRevPhysEducRes.18.010112
Keywords:
FCI Assessment, FCI-CAT, FCI-CAT validity, Force Concept Inventory, concept inventory
Record Creator:
Metadata instance created February 28, 2022 by Lyle Barbato
Record Updated:
September 28, 2023 by Caroline Hall
Last Update
when Cataloged:
February 17, 2022
Other Collections:

ComPADRE is beta testing Citation Styles!

Record Link
AIP Format
J. Yasuda, M. Hull, and N. Mae, , Phys. Rev. Phys. Educ. Res. 18 (1), 010112 (2022), WWW Document, (https://doi.org/10.1103/PhysRevPhysEducRes.18.010112).
AJP/PRST-PER
J. Yasuda, M. Hull, and N. Mae, Improving test security and efficiency of computerized adaptive testing for the Force Concept Inventory, Phys. Rev. Phys. Educ. Res. 18 (1), 010112 (2022), <https://doi.org/10.1103/PhysRevPhysEducRes.18.010112>.
APA Format
Yasuda, J., Hull, M., & Mae, N. (2022, February 17). Improving test security and efficiency of computerized adaptive testing for the Force Concept Inventory. Phys. Rev. Phys. Educ. Res., 18(1), 010112. Retrieved December 12, 2024, from https://doi.org/10.1103/PhysRevPhysEducRes.18.010112
Chicago Format
Yasuda, J, M. Hull, and N. Mae. "Improving test security and efficiency of computerized adaptive testing for the Force Concept Inventory." Phys. Rev. Phys. Educ. Res. 18, no. 1, (February 17, 2022): 010112, https://doi.org/10.1103/PhysRevPhysEducRes.18.010112 (accessed 12 December 2024).
MLA Format
Yasuda, Jun-ichiro, Michael Hull, and Naohiro Mae. "Improving test security and efficiency of computerized adaptive testing for the Force Concept Inventory." Phys. Rev. Phys. Educ. Res. 18.1 (2022): 010112. 12 Dec. 2024 <https://doi.org/10.1103/PhysRevPhysEducRes.18.010112>.
BibTeX Export Format
@article{ Author = "Jun-ichiro Yasuda and Michael Hull and Naohiro Mae", Title = {Improving test security and efficiency of computerized adaptive testing for the Force Concept Inventory}, Journal = {Phys. Rev. Phys. Educ. Res.}, Volume = {18}, Number = {1}, Pages = {010112}, Month = {February}, Year = {2022} }
Refer Export Format

%A Jun-ichiro Yasuda %A Michael Hull %A Naohiro Mae %T Improving test security and efficiency of computerized adaptive testing for the Force Concept Inventory %J Phys. Rev. Phys. Educ. Res. %V 18 %N 1 %D February 17, 2022 %P 010112 %U https://doi.org/10.1103/PhysRevPhysEducRes.18.010112 %O application/pdf

EndNote Export Format

%0 Journal Article %A Yasuda, Jun-ichiro %A Hull, Michael %A Mae, Naohiro %D February 17, 2022 %T Improving test security and efficiency of computerized adaptive testing for the Force Concept Inventory %J Phys. Rev. Phys. Educ. Res. %V 18 %N 1 %P 010112 %8 February 17, 2022 %U https://doi.org/10.1103/PhysRevPhysEducRes.18.010112


Disclaimer: ComPADRE offers citation styles as a guide only. We cannot offer interpretations about citations as this is an automated procedure. Please refer to the style manuals in the Citation Source Information area for clarifications.

Citation Source Information

The AIP Style presented is based on information from the AIP Style Manual.

The APA Style presented is based on information from APA Style.org: Electronic References.

The Chicago Style presented is based on information from Examples of Chicago-Style Documentation.

The MLA Style presented is based on information from the MLA FAQ.

Save to my folders

Contribute

Similar Materials