Detail Page

Physical Review Physics Education Research
written by Jun-ichiro Yasuda, Naohiro Mae, Michael M. Hull, and Masa-aki Taniguchi
As a method to shorten the test time of the Force Concept Inventory (FCI), we suggest the use of computerized adaptive testing (CAT). CAT is the process of administering a test on a computer, with items (i.e., questions) selected based upon the responses of the examinee to prior items. In so doing, the test length can be significantly shortened. As a step to develop a CAT-based version of the FCI (FCI-CAT), we examined the optimal test length of the FCI-CAT such that accuracy and precision [which we measure in terms of bias, standard error, and root-mean-square error (RMSE)] of Cohen's d would be comparable to that of the full FCI for a given class size. First, we estimated the item parameters of the FCI items based on the three-parameter logistic model of item response theory, which are used in the algorithm of CAT. For this estimation, we used 2882 responses of Japanese university students. Second, we conducted a Monte Carlo simulation to analyze how the bias, standard error, and RMSE of Cohen's d depend upon the test length. Third, we conducted a post hoc simulation to examine the consistency of the Monte Carlo results with what would have been obtained using empirical responses. For this comparison, we used 86 pairs of pre- and post- test responses of Japanese university students. As a result, we found that for a class size of 40, we may reduce the test length of the FCI-CAT to 15–19 items, thereby reducing the test time of the FCI to 50%–63%, with an accompanying decrease in accuracy and precision of only 5%–10%. The results of the Monte Carlo study and the post hoc simulation were consistent, which supports the adequacy of our Monte Carlo study and its relevance in terms of administering the FCI-CAT in real classrooms.
Physical Review Physics Education Research: Volume 17, Issue 1, Pages 010115
Subjects Levels Resource Types
Classical Mechanics
- General
Education Foundations
- Assessment
= Instruments
- Research Design & Methodology
= Data
= Validity
- Sample Population
General Physics
- Physics Education Research
- Lower Undergraduate
- Reference Material
= Research study
Intended Users Formats Ratings
- Researchers
- Professional/Practitioners
- application/pdf
- text/html
  • Currently 0.0/5

Want to rate this material?
Login here!


Access Rights:
Free access
License:
This material is released under a Creative Commons Attribution 3.0 license.
Rights Holder:
American Physical Society
DOI:
10.1103/PhysRevPhysEducRes.17.010115
Keywords:
AI-based assessment, FCI validity, adaptive assessment, adaptive learning
Record Creator:
Metadata instance created May 12, 2021 by Bruce Mason
Record Updated:
April 1, 2022 by Caroline Hall
Last Update
when Cataloged:
March 9, 2021
Other Collections:

ComPADRE is beta testing Citation Styles!

Record Link
AIP Format
J. Yasuda, N. Mae, M. Hull, and M. Taniguchi, , Phys. Rev. Phys. Educ. Res. 17 (1), 010115 (2021), WWW Document, (https://doi.org/10.1103/PhysRevPhysEducRes.17.010115).
AJP/PRST-PER
J. Yasuda, N. Mae, M. Hull, and M. Taniguchi, Optimizing the length of computerized adaptive testing for the Force Concept Inventory, Phys. Rev. Phys. Educ. Res. 17 (1), 010115 (2021), <https://doi.org/10.1103/PhysRevPhysEducRes.17.010115>.
APA Format
Yasuda, J., Mae, N., Hull, M., & Taniguchi, M. (2021, March 9). Optimizing the length of computerized adaptive testing for the Force Concept Inventory. Phys. Rev. Phys. Educ. Res., 17(1), 010115. Retrieved March 4, 2024, from https://doi.org/10.1103/PhysRevPhysEducRes.17.010115
Chicago Format
Yasuda, J, N. Mae, M. Hull, and M. Taniguchi. "Optimizing the length of computerized adaptive testing for the Force Concept Inventory." Phys. Rev. Phys. Educ. Res. 17, no. 1, (March 9, 2021): 010115, https://doi.org/10.1103/PhysRevPhysEducRes.17.010115 (accessed 4 March 2024).
MLA Format
Yasuda, Jun-ichiro, Naohiro Mae, Michael Hull, and Masa-aki Taniguchi. "Optimizing the length of computerized adaptive testing for the Force Concept Inventory." Phys. Rev. Phys. Educ. Res. 17.1 (2021): 010115. 4 Mar. 2024 <https://doi.org/10.1103/PhysRevPhysEducRes.17.010115>.
BibTeX Export Format
@article{ Author = "Jun-ichiro Yasuda and Naohiro Mae and Michael Hull and Masa-aki Taniguchi", Title = {Optimizing the length of computerized adaptive testing for the Force Concept Inventory}, Journal = {Phys. Rev. Phys. Educ. Res.}, Volume = {17}, Number = {1}, Pages = {010115}, Month = {March}, Year = {2021} }
Refer Export Format

%A Jun-ichiro Yasuda %A Naohiro Mae %A Michael Hull %A Masa-aki Taniguchi %T Optimizing the length of computerized adaptive testing for the Force Concept Inventory %J Phys. Rev. Phys. Educ. Res. %V 17 %N 1 %D March 9, 2021 %P 010115 %U https://doi.org/10.1103/PhysRevPhysEducRes.17.010115 %O application/pdf

EndNote Export Format

%0 Journal Article %A Yasuda, Jun-ichiro %A Mae, Naohiro %A Hull, Michael %A Taniguchi, Masa-aki %D March 9, 2021 %T Optimizing the length of computerized adaptive testing for the Force Concept Inventory %J Phys. Rev. Phys. Educ. Res. %V 17 %N 1 %P 010115 %8 March 9, 2021 %U https://doi.org/10.1103/PhysRevPhysEducRes.17.010115


Disclaimer: ComPADRE offers citation styles as a guide only. We cannot offer interpretations about citations as this is an automated procedure. Please refer to the style manuals in the Citation Source Information area for clarifications.

Citation Source Information

The AIP Style presented is based on information from the AIP Style Manual.

The APA Style presented is based on information from APA Style.org: Electronic References.

The Chicago Style presented is based on information from Examples of Chicago-Style Documentation.

The MLA Style presented is based on information from the MLA FAQ.

Save to my folders

Contribute

Similar Materials