Design strategies for research-based physics activities

Physics education researchers often design activities, then test whether the activities are effective in helping students learn. Many published activities have been described in the physics education literature. However, some instructors may want to create their own activities; these instructors may want to know about how the published activities were designed. To this end, we have interviewed several prominent designers and analyzed their publications as well as an American Association of Physics Teachers’ report on lab design. This paper focuses on a particular set of design philosophies that were important to these designers. “Revisiting cycles,” which address a single question in depth, were a common theme. We also argue that designers’ view of conceptual learning and of “thinking like a physicist” shapes their design plans.


I. INTRODUCTION
Physics education researchers advise instructors to engage students in activities.In this paper, we consider labs as well as activities with a lab component, and we focus on introductory physics curricula [1][2][3][4][5][6][7][8].Because these activities play a central role in the learning process, researchers have invested considerable effort into optimizing activities and discerning which activities are most effective [9].Our intent in this paper is to understand the design philosophies or strategies emphasized by the activity designers."To design" means "to plan the form and structure of something" [10].We will always use the word in the sense of an instructor or researcher designing an activity.
Individual groups have published accounts of their philosophy of activity design [1][2][3][4][5][6][7][8], but to our knowledge, no comparative analysis has been performed that takes the step of interviewing the designers themselves.One reason that such an analysis is useful is that designers may update their perspectives over time and may not have published everything they know about their activities' design features.Examples of "design features" for activities include students' predictions, generating hypotheses, and thinking about assumptions.
We do not require that activities described in this paper are only a lab; e.g.Modeling Instruction also includes whiteboarding and class discussions [1].
One recent source of information about physics labs is the American Association of Physics Teachers' Recommendations for the Undergraduate Physics Laboratory Curriculum [11].Like the current paper, the AAPT report endorses both "lab only" and "integratedlaboratory" formats.
A second important source of information that we will make use of are the activities themselves [12][13][14][15].We will also refer to publications that lab designers have written about their work [1][2][3][4][5][6][7][8].Finally, we have interviewed the designers of the activities to find out their opinions about what makes their laboratory activities effective.This paper focuses on four major curricula: the Investigative Science Learning Environment (ISLE), Modeling Instruction, RealTime Physics, and Workshop Physics [1][2][3][4][5][6][7][8].In some cases, such as ISLE, multiple curricula are available that implement a particular design philosophy; we have focused on the philosophy as a whole.Although AAPT addresses both introductory and advanced labs, in this paper we consider only introductory labs, since the major published curricula are introductory.
This work is part of a larger project to investigate physics teaching in a studio physics environment.In studio physics, there is no standard curriculum and instructors typically design their own laboratory activities.In this context, we hope to achieve the following goals: G1: To be able to make recommendations to instructors about designing activities.
G2: To be able to examine an instructor-designed activity and characterize it in terms of a standard set of design features or a standard taxonomy of features, in order to better evaluate or give feedback to instructors about their activities.
The present work makes progress toward these goals by organizing information collected from designers of popular activities and from their publications.This information can be used to plan recommendations (G1) or to develop techniques for characterizing activities (G2).To this end, we address the following research questions: RQ1: How do designers talk about design features, what do publications say about design features, and how can we characterize these sources jointly?RQ2: What, if anything, do the designers and their publications say that adds to the recommendations in the AAPT report?

II. LITERATURE REVIEW
Researchers have written extensively about the design of activities and labs.One common theme is the intended outcome or goal of activities.Some authors indicate that the purpose of instruction is to teach "concepts" or "conceptual thinking" [16] or "understanding" and "big ideas" [10].Others argue that labs in particular might not help with content knowledge [17].Instead, "communicating physics" [11], "design skills," or "ability to design and conduct experiments" [18] should be the target (in this instance, "design" means students designing experiments.)In this paper, we will emphasize a goal that is especially important to the AAPT report and mentioned by others as well, namely "thinking like a physicist" or scientist.[10,11].
A second common topic is the importance of student thinking, including "common student ideas" [19], "misunderstandings" [10], or "pre-instruction knowledge" [16].This student thinking can be "elicited" [20] with the right question and then "confronted" [16].This elicit / confront cycle is a special case of "revisiting," which we will discuss in the current paper.

III. METHODS
In this section, we describe how the data from our four sources was collected and analyzed.
When coding publications and interviews, we used a constant comparative approach, involving open coding (freely coding segments of data, inventing new codes, and comparing segments of data with one another) and axial coding (relating the to one another to develop categories) [21].We coded publications produced by the designers of our four activities, including two publications each for ISLE, Modeling Instruction, RealTime Physics, and Workshop Physics [1][2][3][4][5][6][7][8].We chose publications according to three criteria.When possible, the papers should (1) give an overview of the design philosophy rather than simply quantitative evidence that the method works, (2) be recently published, (3) be highly cited.(It was not always possible to find publications that satisfied all three of these conditions simultaneously; the average year of publication is 2009 and the average number of citations is 83.) We also interviewed several designers of the four methods.We interviewed one designer for ISLE, three for Modeling Instruction, one for RealTime Physics, and one for Workshop Physics.We interviewed more designers for Modeling Instruction in part because we obtained more names of designers for this method.Interviews were one hour in length and were semi-structured.In each interview, we asked what design principles or features the designers used to construct their activities.Beyond this question, we asked designers about the purpose of specific design features as well as their opinion of the similarities and differences between their approach and other popular methods.Additional questions varied depending on their responses to our initial questions and the nature of their activity.
We created a separate set of codes from the AAPT report.This set of codes was developed directly from a list of 36 specific recommendations for introductory-level labs that are given in the report.We compared these codes with actual labs in order to ensure we understood them.An example recommendation is that students should consider "assumptions, limitations, and simplifications." Finally, we compared the different codes derived from these sources with one another.These comparisons resulted in multiple types of categories, of which we emphasize two in this paper: first, the issue of "revisiting cycles," and second, the "goals" of activity designers.

IV.REVISITING CYCLES
The idea of a learning "cycle" or "sequence" was especially important to the designers we interviewed and was mentioned in publications about the four methods.Designers as well as publications for three of the four methods mentioned "cycles," while a publication about the fourth method described a learning "sequence."For example, the RealTime Physics designer described a cycle involving prediction, observation, and comparison, stating that "Workshop Physics, Real Time Physics, Interactive Lecture Demonstrations... all three of those very heavily use that learning cycle."Workshop Physics uses a "sequence" of prediction, observation, reflection, theory, and application [5].Likewise, ISLE "encourages students to come up with multiple explanations for the same observational experiment or the same phenomenon, and to test them by ruling them out" according to the ISLE designer.The pattern of observation, explanation, and testing is part of the ISLE cycle [3].A cycle or sequence is composed of two or three design features that work together when combined in a certain order.The word "cycle" implies that the sequence will repeat; however, these "cycles" don't always repeat.For instance, the RealTime Physics designer asserted that some RealTime lessons do not contain the prediction cycle.Thus, the word "sequence" seems appropriate as well.
All of these cycles are examples of "revisiting" cycles, an idea that was described in our previous publication [22].A revisiting cycle is composed of a "visit," where a question is asked of the students, and a "revisit," where the same question is addressed again in another way.For instance, "predict/observe/compare" is revisiting because the prediction is the "visit" and the observation/comparison is the "revisit."The prediction and observation are different ways to answer the same question.
Apart from predict/observe/compare, two significant examples of revisiting are the ISLE and Modeling cycles: 1.The ISLE cycle can vary in length, but it often takes place on a larger scale than a single prediction/experiment.In particular, an ISLE cycle may include three distinct types of experiments: observation, testing, and application experiments.The student makes an observation, then tries to explain a phenomenon (the "visit") and then tests their explanation (the "revisit.")For example, the ISLE designer noted that chapter four of the Active Learning Guide [12] begins with observation and testing experiments (4.1.2,4.1.3)and ends with application (4.4.8).
The AAPT list of recommendations indicates one example that hints at an ISLE-like revisiting cycle: "Students should be provided multiple instances during the four-year curriculum where they clearly see the entire cycle from asking a question to deciding between alternate explanations or models based on observation." 2. Modeling involves multiple cycles.The learning cycle in Modeling is not revisiting, as it is centered on a model or representation rather than a single question or problem.However, a Modeling designer noted that "within each learning cycle you see lots of little … activity cycles, which are, you know, 'do something in a small group, write about it on a small whiteboard, bring the small whiteboard to the group meeting and share that and discuss it.'"The "something" or "that" which is considered, written about, shared, and then discussed is a common question or problem; therefore, this is revisiting.

V.GOALS OF DESIGN
We also compare the goals of the designers we interviewed those of the AAPT report.We focus on two types of goals: "thinking like a physicist" and content knowledge.The AAPT report states that: "In this laboratory guidelines document, thinking like a physicist and constructing knowledge pervade all of the specific lab goals articulated since the enterprise of physics is the construction of new knowledge [emphasis added]." A. Thinking like a physicist or scientist AAPT in one instance hints that thinking like a physicist implies "constructing knowledge that does not rely on an outside authority."However, they also say that "the laboratory should contain experiences that explicitly support a department's goals for helping students think like physicists as it is locally conceived [emphasis added]."In other words, AAPT hopes to support the scientific vision of each individual department.Our interview subjects also cared about "thinking like a physicist or scientist."However, they had very specific ideas about what it meant to "think like a physicist."We argue that these specific ideas motivate the revisiting cycles and design philosophies for ISLE and Modeling Instruction.
The ISLE designer said that "you are thinking like a physicist in your life already.If you look at little children, they actually do ISLE all the time."Specifically, "They would come up with hypotheses to explain evidence, with multiple hypotheses, and then they would actually systematically test them without saying "I'm testing a hypothesis."This, for the ISLE designer, is what it means to think like a physicist; and this mirrors the ISLE revisiting cycle, which is built around generating and testing hypotheses.
A Modeling designer gave a somewhat different perspective.This designer said: "we have been developing using sort of a Modeling cycle which is based on a learning cycle approach, but the idea being that the epistemological background is that we believe that science proceeds through the building, validation, and testing of models."This vision of science justifies the Modeling approach by highlighting the role of models.

B. Conceptual learning
Another difference is that our designers emphasized conceptual learning, especially in RealTime and Workshop Physics: "these hands on [activities] are directed at trying to overcome conceptions students tend to bring to a course that makes it difficult to learn" [Workshop Physics designer].Although conceptual learning was significant in ISLE and Modeling Instruction, it was secondary: for example, "The goal of Modeling is to use these models and in order to do that you have to … stop by some of these other things as well.You know, conceptual reasoning, active learning, these sorts of things" [Modeling designer].Likewise, "the main goal of ISLE is to help students learn to think like physicists, [but] we use traditional PER assessments as well" [2].
The AAPT report frequently mentions "constructing knowledge," which is associated with the ability to "collect, analyze, and interpret real data from personal observations of the physical world to develop a physical worldview."It is not clear how important conceptual understanding is to the "physical worldview."As Wieman and Holmes point out, the report does not emphasize conceptual learning [17].Specific theories of student conceptualization, such as "preconceptions," are not mentioned, and the word "concept" appears only a few times in the report.We suggest that the lessened emphasis on conceptual understanding might be part of the reason that the predict/observe/compare cycle is not mentioned in the 36 recommendations.("Prediction" is mentioned, but no cycle involving prediction is described.)

VI.CONCLUSION AND DISCUSSION
With regard to our research question RQ1 (what do designers and publications say about design features?),we have emphasized two distinct sets of categories: (1) revisiting cycles and (2) design goals.In response to our goal G1 (making recommendations), we suggest that instructors should be aware of the questions that are challenging to their students and use established "revisiting" strategies, such as predict/observe/compare, to help students reconsider these questions [22].They should also consider imitating known models for "thinking like a physicist," as in ISLE or Modeling.In order to accomplish goal G2 (characterizing activities), our results suggest we should look for ISLE-and Modeling-like revisiting sequences in other activities.
With regard to question RQ2, we suggest that our designers differ from the AAPT report in that they provide specific, idiosyncratic ideas of what it means to think like a physicist.These specific ideas led them to propose particular revisiting cycles.In contrast, the AAPT report is open to a variety of "local" perspectives.
It is also the case that our four methods are not only labs they include "board meetings," homework, may take up six hours per week, etc. We argue that this cannot explain the four methods' emphasis on revisiting cycles; predict/observe/compare and the ISLE cycle can fit equally well into a single lab or a six hour per week studio class.
At present, we have limited information as to whether a physics activity can be effective without a revisiting cycle, since major design philosophies tend to include these cycles.This is true whether the goals have to do with concepts, skills, or "thinking like a physicist."However, some published activities lack revisiting [22].We suggest that researchers should find out whether (and why) these revisiting-free activities are still effective.We also hope to study additional curricula in future works.
edited by Ding, Traxler, and Cao; Peer-reviewed, doi:10.1119/perc.2017.pr.098Published by the American Association of Physics Teachers under a Creative Commons Attribution 3.0 license.Further distribution must maintain attribution to the article's authors, title, proceedings citation, and DOI.2017 PERC Proceedings,