Supported Site Ball State University: Assessment

Algebra-based Introductory Mechanics: FCI
Semester N Pretest ave. score (%) Posttest ave. score (%) Hake gain
Fall, 2003 24 39.9 +/- 3.3 58.2+/- 3.5 0.32 +/- .04
Spring, 2004 42 33.3 +/- 2.4 52.2 +/- 2.5 0.28 +/- .05
Fall, 2004 38 26.6 +/- 2.0 54.2+/- 3.1 0.38 +/- .04
Fall, 2005 32 28.6 +/- 2.2 45.9+/- 3.1 0.25 +/- .04
Fall, 2006 33 31.9 +/- 2.4 51.5 +/- 2.5 0.29 +/- .05
Spring, 2007; 44 26.8 +/- 2.5 47.1 +/- 2.5 0.28 +/- .05

N = number of matched pairs of students

  • The data provided above is from the only Ball State physics faculty member who has taught PhysTEC courses from the beginning and from whom the most data has been gathered. This instructor used interactive engagement methods.
Algebra-based E&M, CSEM
Semester N Pretest ave. score (%) Posttest ave. score (%) Hake gain
Spring, 2004 21 21.0 +/- 3.4 33.6 +/- 3.6 0.16 +/- .05
Spring, 2005 39 20.5 +/- 2.5 35.3 +/- 2.7 0.19 +/- .05
Spring, 2006 36 20.1 +/- 2.6 28.7 +/- 2.8 0.14 +/- .05
Fall, 2006 27 22.5 +/- 3.2 38.4 +/- 3.2 0.22 +/- .05

N = number of matched pairs of students

Calculus-based Introductory Mechanics: FCI
Semester N Pretest ave. score (%) Posttest ave. score (%) Hake gain
Fall, 2007 29 40.9 +/- 3.2 62.6+/- 4.0 0.37 +/- .09
Spring, 2008 13 40.0 +/- 5.6 64.4 +/- 6.7 0.41 +/- .08

N = number of matched pairs of students

  • For the past several years, FCI data have been gathered from the introductory algebra-based course at the question level and at the student level (more than 700 students). For further discussion, please see Appendix 1-A below.
  • Analysis of pre-and post-tests on the FCI and CSEM for the department's conceptual, elementary education conceptual, algebra-based, and calculus-based courses have shown that the PhysTEC project has positively impacted other course reforms over the course of the project.
Reformed Teacher Observation Protocol (RTOP) evaluation of two BSU PhysTEC Teachers
Course taught Instructor name Score(s) Date(s) of observation
High school physics, 1st semester Teacher 1 82 & 70 Oct & May, 2007
High school physics, 1st semester Teacher 2 64 Feb, 2008
  • The observer noted that on the second observation of Instructor 1, “there was a lab that day which was pretty scripted, so that’s the reason for a fairly low score as compared to previous observation.” This situation provides an example of the difficulty of interpretation with a small number of observations.

Physics for Elementary Teachers (PET)

  • In 2004-2005, a comparison study of Ball State Elementary Education majors was made using the PET curriculum (Robert Hill, instructor) and large class section curriculum (James Watson, instructor).
  • The PET analyses on the PET diagnostic exams were conducted by the Mallinson Institute for Science Education at Western Michigan University.

Appendix 1: Assessment

Appendix 1-A: Analysis of FCI Assessment Data – Instruction Strategies and Materials

For the past several years, FCI data have been gathered from the introductory algebra-based course at the question level and at the student level (more than 700 students). Results of this analysis have shown that the FCI data prior to instruction indicate that the percentage of questions with correct answers ranged from 2% to 65%; nine (9) of the 30 questions had 15% correct or less in the pretest, where 20% is consistent with random guessing. Therefore, this data is quite useful to instructors in assessing the misconceptions brought to their courses by students. An analysis of the posttest data indicate that additional intervention strategies are still needed to improve student knowledge of physical concepts.

Further analyses of the FCI data also indicated that the normalized gain was correlated with the preinstruction score of the students. This finding is in disagreement with earlier results by Hake, where the conclusion was that there was no correlation between normalized gain and preinstruction FCI scores. These results are also consistent with the published results by Coletta and Phillips, Am. J. Phys. 73, 1172 (2005).

Appendix 1-B: Multivariable Analysis Involving FCI scores And Other Data

A new multivariable analysis has been started by Dave Ober that involves the FCI scores and many other data, such as instructor, class, and student major. The new analysis includes new variables, such as a student’s high school, gender, and SAT math score. It should be noted that even more high school data can be found, such as the average socio-economic status, by knowing the high school that a student had attended. Correlations were sought between each of these variables, and which of the many variables can influence the FCI and CSEM scores. This analysis is ongoing, but preliminary results indicate that there may be a strong instructor influence on the scores.

Appendix 1-C: Analysis of Physics Enrollments in Schools with Physics Teachers Who Graduated from BSU

Another current analysis has looked into the numbers of students in selected high schools that are taking physics. Of particular interest are schools that have physics teachers who are Ball State graduates. Preliminary results indicate that the numbers of high school students taking physics in these schools has increased since the BSU graduates have become teachers at those schools. This increase hints that our graduates are effective teachers in the classroom—otherwise the physics enrollments would drop.

Appendix 1-D: Pedagogy of BSU Science Preservice Teachers

Part a:

The pedagogy of BSU preservice students was analyzed using the Interstate New Teacher Assessment and Consortium (INTASC) Principles. Two of these principles were used to assess the inquiry performance of preservice teachers. In a study by 2006-2007 TIR Elaine Gwinn, it was found that the science students exhibited higher performance scores than students of other content disciplines at Ball State University.

Part b:

Ball State University student teaching supervisors were able to provide a database that provides assessment instruments for secondary science teachers. These instruments determine the effectiveness of the science methods classes. These assessment instruments are based on the INTASC principles, adopted both by Ball State University and the Indiana Professional Standards Board.

Appendix 1-E: Effect of PhysTEC on Other Course Reforms at BSU

Analysis of pre-and post-tests on the FCI and CSEM for the department's conceptual, elementary education conceptual, algebra-based, and calculus-based courses have shown that the PhysTEC project has positively impacted other course reforms over the course of the project.