Skip to main content
SpringerLink
Log in

Measuring Model-Based High School Science Instruction: Development and Application of a Student Survey

  • Published:
Journal of Science Education and Technology Aims and scope Submit manuscript

Abstract

This study tested a student survey to detect differences in instruction between teachers in a modeling-based science program and comparison group teachers. The Instructional Activities Survey measured teachers’ frequency of modeling, inquiry, and lecture instruction. Factor analysis and Rasch modeling identified three subscales, Modeling and Reflecting, Communicating and Relating, and Investigative Inquiry. As predicted, treatment group teachers engaged in modeling and inquiry instruction more than comparison teachers, with effect sizes between 0.55 and 1.25. This study demonstrates the utility of student report data in measuring teachers’ classroom practices and in evaluating outcomes of a professional development program.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Adamson SL, Banks D, Burtch M, Cox F III, Judson E, Turley JB, Benford R, Lawson AE (2003) Reformed undergraduate instruction and its subsequent impact on secondary school teaching practice and student achievement. J Res Sci Teach 40(10):939–957

    Article  Google Scholar 

  • Aleamoni LM (1999) Student rating myths versus research facts from 1924 to 1998. J Personnel Eval Educ 13:153–166

    Article  Google Scholar 

  • Akerson VL, Abd-El-Khalick F, Lederman NG (2000) Influence of a reflective explicit activity-based approach on elementary teachers’ conceptions of Nature of Science. J Res Sci Teach 37:295–317

    Article  Google Scholar 

  • Andrich D (1978) A rating formulation for ordered response categories. Psychometrika 43:561–573

    Article  Google Scholar 

  • Association of American Universities (1995) Survey of undergraduate education activities. Author, Washington, DC

    Google Scholar 

  • Banilower ER (2005) A study of the predictive validity of the LSC classroom observation protocol. Horizon Research, Inc., Chapel Hill

    Google Scholar 

  • Beeth ME, Hewson PW (1999) Learning goals in an exemplary science teacher’s practice: cognitive and social factors in teaching for conceptual change. Sci Educ 83:738–760

    Article  Google Scholar 

  • Bill & Melinda Gates Foundation (2010a) Learning about teaching—initial findings from the measures of effective teaching project. Author, Seattle. Accessed November 1, 2010 from http://www.metproject.org/reading

  • Bill & Melinda Gates Foundation (2010b) Working with teachers to develop fair and reliable measures of effective teaching: framing paper of the measures of effective teaching project. Author, Seattle. Accessed November 1, 2010 from http://www.metproject.org/reading

  • Bond TG, Fox CM (2001) Applying the Rasch model: fundamental measurement in the human sciences, 2nd edn. Routledge, New York

    Google Scholar 

  • Brewe E, Sawtelle V, Kramer LH, O’Brien GE, Rodriguez I, Pamelá P (2010) Toward equity through participation in Modeling Instruction in introductory university physics. Phys Rev Special Topics Phys Educ Res 6:010106

    Google Scholar 

  • Clement J (1989) Learning via model construction and criticism. In: Glover G, Ronning R, Reynolds C (eds.) Handbook of creativity: assessment, theory and research. Plenum, New York, NY, pp 341–381

    Google Scholar 

  • Clement J (2008) Six levels of organization for curriculum design and teaching. In: Clement J, Rea-Ramirez MA (eds.) Model based learning and instruction in science. Springer, New York, NY, pp 255–272

    Chapter  Google Scholar 

  • Cohen J (1988) Statistical power analysis for the behavioral sciences, 2nd edn. Lawrence Erlbaum Associates, Mahwah

    Google Scholar 

  • Desimone LM (2009) Improving impact studies of teachers’ professional development: toward better conceptualizations and measures. Educ Res 38(3):181–199

    Google Scholar 

  • Desimone L, Porter AC, Garet MS, Yoon KS, Birman BF (2002) Effects of professional development on teachers’ instruction: results from a three-year longitudinal study. Educ Eval Policy Anal 24(2):81–112

    Article  Google Scholar 

  • Duran E, Duran LB (2005) Project ASTER: a model staff development program and its impact on early childhood teachers’ self-efficacy. J Element Sci Educ 17(2):1–12

    Article  Google Scholar 

  • Fenstermacher GD, Richardson V (2005) On making determinations of quality in teaching. Teach College Rec 107(1):186–213

    Article  Google Scholar 

  • Floyd FJ, Widaman KF (1995) Factor analysis in the development and refinement of clinical assessment instruments. Psychol Assess 7:286–299. doi:10.1037/1040-3590.7.3.286

    Article  Google Scholar 

  • Fulmer GW (2008) Successes and setbacks in collaboration for science instruction: connecting teacher practices to contextual pressures and student learning outcomes. VDM Publishing, Saarbrücken, Germany

  • Fulmer GW (2011) Estimating critical values for strength of alignment among curriculum, assessments, and instruction. J Educ Behav Stat 36(3):381–402. doi:10.3102/1076998610381397

    Article  Google Scholar 

  • Garet MS, Porter AC, Desimone L, Birman BF, Yoon KS (2001) What makes professional development effective? Results from a national sample of teachers. Am Educ Res J 38:915–945

    Article  Google Scholar 

  • Hestenes D, Wells M, Swackhamer G (1992) Force concept inventory. Phys Teach 30:141–158

    Article  Google Scholar 

  • Ihaka R, Gentleman R (1996) R: a language for data analysis and graphics. J Comput Graph Stat 5(3):299–314

    Google Scholar 

  • Liang LL, Fulmer GW, Majerich DM, Clevenstine R, Howanski R (2012) The effects of a model-based physics curriculum program with a physics first approach: a causal-comparative study. J Sci Educ Technol 21(1):114–124. doi:10.1007/s10956-011-9287-2

    Article  Google Scholar 

  • Liu X, Zhang BH, Liang LL, Fulmer GW, Kim B, Yuan HQ (2009) Alignment between the physics content standards and standardized tests: a comparison among US-NY, Singapore, and China-Jiangsu. Sci Educ 93:777–797. doi:10.1002/sce.20330

    Article  Google Scholar 

  • Johnson DK, Duvernoy R, McGill P, Will JF (1996) Educating teachers together: teachers as learners, talkers, and collaborators. Theory Practice 35(3):173–178

    Google Scholar 

  • Johnson CC, Kahle JB, Fargo JD (2007) A study of the effect of sustained, whole-school professional development on student achievement in science. J Res Sci Teach 44(6):775–786

    Article  Google Scholar 

  • Judson E, Lawson AE (2007) What is the role of constructivist teachers within faculty communication networks? J Res Sci Teach 44(3):490–505. doi:10.1002/tea.20117

    Article  Google Scholar 

  • Karplus R (1977) Science teaching and the development of reasoning. J Res Sci Teach 14:169–175

    Article  Google Scholar 

  • Koziol SM Jr, Burns P (1986) Teachers’ accuracy in self-reporting about instructional practices using a focused self-report inventory. J Educ Res 79(4):205–209

    Google Scholar 

  • Lawley DN, Maxwell AE (1962) Factor analysis as a statistical method. J R Stat Soc Ser D (The Statistician) 12(3):209–229

    Google Scholar 

  • Lawrenz F, Wood NB, Kirchhoff A, Kim NK, Eisenkraft A (2009) Variables affecting physics achievement. J Res Sci Teach 44(6):775–786

    Google Scholar 

  • Levine A (2006) Educating school teachers. Education Schools Project, New York

    Google Scholar 

  • Lieberman A (2000) Networks as learning communities: shaping the future of teacher development. Teach Educ 51:221–227

    Article  Google Scholar 

  • Linacre JM (2007) WINSTEPS (Version 3.61.2) [Computer Software]. Winsteps.com, Chicago

  • Marsh HW (1984) Students’ evaluations of university teaching: dimensionality, reliability, validity, potential biases, and utility. J Educ Psychol 76(5):707–754

    Article  Google Scholar 

  • Minner DD, Levy AJ, Century J (2010) Inquiry-based science instruction—what is it and does it matter? Results from a research synthesis years 1984 to 2002. J Res Sci Teach 47(4):474–496

    Article  Google Scholar 

  • National Research Council (NRC) (1996) National science education standards. National Academy Press, Washington, D.C.

    Google Scholar 

  • OECD (2009) The Rasch model. In: OECD (author) PISA data analysis manual: SPSS (2nd Ed.). OECD Publishing, Paris, doi:10.1787/9789264056275-6-en

  • Ohio Department of Education (2011) Ohio achievement assessments May 2011 Administration: statistical summary. Author, Columbus. Retrieved 1 December 2011 from http://www.ode.state.oh.us/GD/DocumentManagement/DocumentDownload.aspx?DocumentID=107732/

  • Park S, Jang J-Y, Chen Y-C, Jung J (2011) Is pedagogical content knowledge (PCK) necessary for reformed science teaching? Evidence from an empirical study. Res Sci Educ 41(2):245–260. doi:10.1007/s11165-009-9163-8

    Article  Google Scholar 

  • Peterson KD, Wahlquist C, Bone K (2000) Student surveys for school teacher evaluation. J Personnel Eval Educ 14(2):135–153

    Article  Google Scholar 

  • Peterson KD (2000) Teacher evaluation: a comprehensive guide to new directions and practices, 2nd edn. Corwin Press, Thousand Oaks

    Google Scholar 

  • Porter AC (2002) Measuring the content of instruction: uses in research and practice. Educ Res 31(7):3–14

    Google Scholar 

  • Raudenbush SW, Bryk AS (2002) Hierarchical linear models: applications and data analysis methods, 2nd edn. Sage, Newbury Park, CA

    Google Scholar 

  • Sawada D, Piburn M, Judson E, Turley J, Falconer K, Benford R, Bloom I (2002) Measuring reform practices in science and mathematics classrooms: the reformed teaching observation protocol. School Sci Math 102(6):245–253

    Article  Google Scholar 

  • Schön DA (1983) The reflective practitioner: how professionals think in action. Basic Books, New York

    Google Scholar 

  • Schwarz CV, White BY (2005) Metamodeling knowledge: developing students’ understanding of scientific modeling. Cogn Instruct 23:165–205

    Article  Google Scholar 

  • Schwarz CV, Gwekwerere YN (2007) Using a guided inquiry and modeling instructional framework (EIMA) to support preservice K-8 science teaching. Sci Educ 91:158–186

    Article  Google Scholar 

  • SERVE (2006) CAPE evaluation framework: looking for technology integration (LoFTI). SERVE Center-UNC Greensboro. Retrieved August, 2009, from http://www.serve.org/Evaluation/Capacity/EvalFramework/resources/LoFTI.php

  • Texas Education Agency (2005) Technical digest for the academic year 2004-2005. A Collaborative Effort of the Texas Education Agency, Pearson Educational Measurement, Harcourt Educational Measurement, and Beck Evaluation and Testing Associates, Inc. Author, Austin. Retrieved 1 December 2011 from http://www.tea.state.tx.us/student.assessment/

  • Van Driel JH, Verloop N (1999) Teachers’ knowledge of models and modeling in science. Int J Sci Educ 21:1141–1153

    Article  Google Scholar 

  • van Driel JH, Beijaard D, Verloop N (2001) Professional development and reform in science education: the role of teachers’ practical knowledge. J Res Sci Teach 38:137–158

    Article  Google Scholar 

  • van Prooijen J-W, van der Kloot WA (2001) Confirmatory analysis of exploratively obtained factor structures. Educ Psychol Measur 61:777–792

    Article  Google Scholar 

  • Venville G, Sheffield R, Rennie LJ, Wallace J (2008) The writing on the wall: classroom context, curriculum implementation, and student learning in integrated, community-based science projects. J Res Sci Teach 45(8):857–880

    Article  Google Scholar 

  • Vescio V, Rossa D, Adams A (2008) A review of research on the impact of professional learning communities on teaching practice and student learning. Teach Teacher Educ 24(1):80–91

    Article  Google Scholar 

  • Vesenka J, Beach P, Munoz G, Judd F, Key R (2002) A comparison between traditional and “modeling” approaches to undergraduate physics instruction at two universities with implications for improving physics teacher preparation. J Phys Teacher Educ 1(1):3–7. Retrieved March 2, 2007, from http://phy.ilstu.edu:16080/jpteo/issues/june2002.html

  • Waight N, Abd-El-Khalick F (2007) The impact of technology on the enactment of “inquiry” in a technology enthusiast’s sixth grade science classroom. J Res Sci Teach 44(1):154–182

    Article  Google Scholar 

  • Webster-Wright A (2009) Reframing professional development through understanding authentic professional learning. Rev Educ Res 79(2):702–739. doi:10.3102/0034654308330970

    Article  Google Scholar 

  • Wells M, Hestenes D, Swackhamer G (1995) A modeling method for high school physics instruction. Am J Phys 63:606–619

    Article  Google Scholar 

  • Wilson CD, Taylor JA, Kowalski SM, Carlson J (2010) The relative effects and equity of inquiry-based and commonplace science teaching on students’ knowledge, reasoning, and argumentation. J Res Sci Teach 47(3):276–301

    Google Scholar 

  • Zeichner K, Liston DP (2006) Teaching student teachers to reflect. In: Hartley D, Whitehead M (eds) Teacher education: professionalism, social justice and teacher education (Volume IV). Routledge, New York, pp 5–34

    Google Scholar 

Download references

Acknowledgments

This work was supported in part by a grant from the National Science Foundation (NSF; award number DUE 03-14806). Additionally, portions of this work were supported by an NSF Independent Research and Development (IR/D) project while the first author was under employment of the NSF. Any opinions expressed are those of the authors, and do not necessarily reflect the views or policies of the NSF.

Author information

Authors and Affiliations

  1. Curriculum, Teaching and Learning Academic Group, National Institute of Education, 1 Nanyang Walk, Singapore, 637616, Singapore

    Gavin W. Fulmer

  2. La Salle University, 1900 W. Olney Ave., Philadelphia, PA, 19141, USA

    Ling L. Liang

Authors
  1. Gavin W. Fulmer
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ling L. Liang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Gavin W. Fulmer.

Appendix: Instructional Activities in Science (Student Form)

Appendix: Instructional Activities in Science (Student Form)

See Table 5.

Table 5 How often do you do the following in this science class?
Full size table

Rights and permissions

Reprints and permissions

About this article

Cite this article

Fulmer, G.W., Liang, L.L. Measuring Model-Based High School Science Instruction: Development and Application of a Student Survey. J Sci Educ Technol 22, 37–46 (2013). https://doi.org/10.1007/s10956-012-9374-z

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10956-012-9374-z

Keywords

Search

Navigation