Learning goals and perceived irrelevance to major within life science majors in introductory physics

In a previous study, students' self-expressed learning orientations towards an exercise centered on self-monitoring one's ability to solve a pre-lab physics problem were identified from a post-test feedback survey given to an introductory algebra-based physics student population spanning six measured semesters, and examined as a potential variable in course performance, force and motion conceptual understanding, and attitudes towards learning physics. The sampled population, which primarily consists of life science majors, was also asked in the same feedback survey to discuss what portion or portions of the course were relevant to their respective choices of major. In this study, we examine the fact that about 50 students out of 218 sampled students, or 23% of the sample population, explicitly stated that they perceived no relevance at all of the course to their respective majors, whereas the other 168 students cited portions of the course or the entirety of the course as being relevant to their majors. A follow-up investigation of perceived relevance versus irrelevance shows that attitudes towards physics will experience more expert-like shifts for students who perceive relevance than students who do not; in particular, the attitudinal survey's item clusters that pertain to personal interest and real-world connections appear to show the strongest effect. Further examination showed that biology majors and health science majors (two distinctive sub-populations of life science majors) show similar pre-post trends for relevance vs. irrelevance perceptions, whereas students with a performance achievement goal appeared to bifurcate between a novice-like shift for perceived irrelevance and no attitudinal shift from pre to post for perceived relevance. Discussion includes emphasis on limitations imposed by institution type among other factors.


I. INTRODUCTION
The notion of relevance of a university-level course to a student [1] applies to non-physics majors' perceptions of how a physics course is relevant to their major, e.g., an introductory algebra-based physics course sequence whose population is predominately life-science. Recent efforts to transform introductory physics for life sciences (IPLS) courses have included discussing relevance via context of physics topics to specific life science populations, e.g., for biology majors [2] or for health science majors (e.g., physical therapy pre-professionals) [3]. However, not all IPLS populations may be treated as monolithic specific majors, e.g., a course section that has a roughly equal mixture of biology majors and health science majors, which may occur in an institution type that differs in size and/or scope from the large research universities typically featured in IPLS publications.
The topic of student mindset, primarily with regard to student beliefs about intelligence, has a very strong presence in psychology literature [4,5], and may contribute to the conversation of perceived relevance to students. Aspects of student mindset which treat relevance have recently become an explicit topic of interest by PER literature, e.g., student responses about challenge [6], establishing a STEM "identity" [7], or learning and achievement goals [8]. The latter item has been a concern with regard to recognizing achievement goals in the form of learning orientations, which were previously indicated to be a factor in the success of upper-level graduate students in physical science programs [9], but not necessarily at the introductory level [8].
Learning goal orientations for life science majors in an algebra-based introductory physics course were identified via surveying students' perception of the utility of a pre-lab problem solving exercise [8]. This exercise was based upon research in "self-diagnosis" metacognition in physics [10,11] with the use of context-rich problems [12] and the assistance of the instructor and lab partners. Students were asked to solve context-rich problems during the first portion of their lab periods, then use a scoring rubric to determine their strengths and weaknesses during the problem-solving attempt, prior to carrying out a lab exercise which was conceptually linked to the pre-lab exercise. The results in Ref. 8 indicated that learning goals were identifiable from students' short-answer written responses to a feedback survey question on the problem solving exercise, and moreover that they appeared to be validated in terms of representing differences in attitudinal gains on problem-solving item clusters within the CLASS survey [13], as well as on conceptual understanding clusters and over the entire survey.
However, this correlation did not translate to any perceivable effect on pre-post measurements for the FCI [14] or on overall course performance.

A. Current Research Goals
In this follow-up study, we supplement the study in Ref. 8 by examining another question given to students on the same post-test feedback survey: "Which sections of the [course] material did you find related well to the coursework in your major?" A non-trivial percentage of student respondents expressed that the course in fact had nothing to do with their majors; they perceived the course as irrelevant. This course section provides an excellent opportunity to consider relevance in a straightforward fashionnamely, whether students who have already committed to a STEM discipline for a major see any relevance of physics to their own choice of major.
Therefore, our first research question is to determine how many biology majors and health science majors, respectively, viewed an introductory physics course as irrelevant to their course major, and to see if this factor affects pre-post measurements within each choice of major. Our second question is to similarly consider whether each subgroup of learning orientations identified in Ref. 8 had a significant influence of student perception of irrelevance, and to what extent these subgroups of students may affect statistical results.

II. PROCEDURE
The sample consisted of 218 students, distributed over six semesters' worth of introductory first-semester algebra-based physics course sections, all with the same course instructor and lecture/laboratory course format. The criteria for inclusion of data from students was that students must a) complete and pass the course, b) submit completed pre-post data (described below) as part of required coursework, and c) not observably give unserious answers on any survey data (usually per a CLASS item that exists for that purpose). Students were given full points for participation, so as to avoid making their responses grade-dependent.
During the last laboratory session of the semester for an introductory algebra-based physics course at a primarily undergraduate institution, students submitted an in-class written response to the following post-test survey question regarding the aforementioned pre-lab metacognitive problem solving exercise: "In what ways did you find this exercise useful towards learning the material in the course?" Three learning goal orientation groups were determined by inter-rater reliability of student responses to this question (see Ref. 8 for details): interest in mastery of a problem solving framework per se, i.e., "Framework" orientation; interest in doing well in terms of course points, i.e., "Performance" orientation; and students who did not express a learning goal, but focused on collaborative process or other items, i.e., "Vague" orientation.
Students also answered a follow-up question: "Which sections of the [class] material did you find related well to the coursework in your major?" In this data set, 50 out of 218 students, or about 23% of the course population, declared that nothing within the course related to their majors, thus expressing a perception of irrelevance; reasons for this statement included, but were not limited to, statements that one's major wasn't really a science major to begin with (e.g., describing one's health administration major as being primarily a business major), or stating that one's science major wasn't really involved with physics (e.g., a preveterinarian biology major not perceiving relevance of physics towards animal health). The remaining students (168 total) cited either a specific physics topic, a specific skill emphasized by the course (e.g., problem solving, conceptual topics, real-world applications), or a specific application to their majors from the course as being relevant to their majors (e.g., torque as applied to physical therapy of limbs).
The CLASS survey [13] and FCI [14] were administered in pre-post format on the first and last weekly laboratory sessions of the semester, respectively. In this paper, we will examine overall FCI performance, overall CLASS performance, and analysis of specific item clusters on the CLASS (as certain item clusters, e.g., Personal Interest and Real-World Context, consider the perception of relevance of physics to the respondent's own life experience). Table 1 shows the distributions of students who saw the physics course as relevant, and students who saw the course as irrelevant. The distributions are shown in terms of the overall student pool; in terms of the subgroups based upon choice of major; and in terms of "Framework," "Performance," and "Vague" learning orientations of students towards the pre-lab problemsolving exercise [8]. Biology majors were slightly less likely to perceive irrelevance than were health science majors, and Vague-oriented students were slightly more likely to perceive irrelevance than were other learning orientations. However, there is no statistical separation between groups for either majors or orientations. Table 2 examines the FCI pre-test, post-test, and normalized gain average for the "Relevant" population versus the "Irrelevant" population. A check of t-tests between groups, as well as a Cohen's d calculation for effect size, are presented. There are no statistically significant differences on the FCI between pretest scores (which are close to random-chance), post-test scores, or normalized gains.

B. Effect of Relevance on FCI Performance
An examination of relevance vs. irrelevance within biology and health science majors did not show any statistical differences in force concept understanding between the two sets of life science majors. A similar examination for relevance vs. irrelevance within learning goal orientations (Framework, Performance, and Vague) showed no statistical significance on pretest scores, posttest scores, or normalized gains for any orientation. A moderate effect size (d ~ 0.3) appeared to show a relatively better score for relevant-perceiving students than for irrelevance-perceiving students within the Framework-oriented and Vague-oriented groups. Table 3 examines the CLASS pre-test, post-test, and normalized gain average for the "Relevant" population versus the "Irrelevant" population, in terms of the overall survey results and in terms of each item cluster within the CLASS. We include t-test comparisons for statistical significance and Cohen's d value for effect size. All comparisons featured were determined to have statistically equal variances via F-test.

C. Effect of Relevance on CLASS Performance
We find that perception of relevance does not appear to be a statistically significant factor on the pretest,  there is a significant difference on the posttest, both overall and within almost every item cluster, with moderate effect sizes according to Cohen's d calculations, and in all cases with a more expert-like view of physics expressed by the students who perceived "Relevance" than by the students who perceived "Irrelevance." There is also a statistical significance in normalized gains, as well as a moderate effect size difference, for the overall CLASS. Students who perceived "Relevance" to the course experienced a slight expertlike shift, while students who perceived "Irrelevance" expressed a slight novice-like shift. This pattern of a more novice-like shift for an "Irrelevance" perception appeared to be somewhat demonstrated across all item clusters, although not always with statistical significance. In particular, the three item clusters that demonstrated the strongest differences between "relevant" and "irrelevant" beliefs were Personal Interest (PI), Real-World Connection (RWC), and Conceptual Understanding (CU). The first two clusters are of significance because they do not necessarily reflect the pre-lab problem solving exercise about which the students were specifically asked; they are more directly related to explicit relevance concerns within student responses. Even so, the effect sizes for these findings are only moderate (d ~ 0.33 in each case).
For a closer examination, we consider the CLASS results within each choice of life science major and within each learning orientation. Table 4 shows the overall CLASS results for choice of life science major, and Table 5 shows overall CLASS results for learning orientations. Table 4 indicates that each life science major separately exhibits the pattern within the overall CLASS results in Table 3, between relevant-perceiving and irrelevant-perceiving students. Both sets of majors show no statistical significance on the pretest, but a large effect size and significance on the posttest (with more expert-like scores from relevant-perceiving students in both cases).
There is also a large effect size and statistical significance for normalized gains within the biology majors, again with a novice-like shift for irrelevantperceiving students and a slight expert-like shift for relevant-perceiving students. This pattern also appears for health science majors' gains, but is not statistically significant in this case. With regard to CLASS item clusters, biology majors who perceived relevance had statistically higher post-test scores and normalized gains on the PI and RWC clusters, as well as on the Sense Making-Effort (SME) cluster, with large effect sizes for  Table 5 shows that the difference between relevant and irrelevant perceptions appears to particularly matter for Performance-oriented students' attitudinal shifts (presented in terms of normalized gains on the percentage of expert-like responses); students who perceive relevance experience little change on average, but students who perceive irrelevance have a rather noticeable novice-like shift difference (with statistical significance and a large effect size). There was no similar effect within framework-oriented students (who had a slight expert-like gain in either case) or within vague-oriented students (who had a slight novice-like gain in either case). Within the item clusters, the Performance-oriented students had statistical significance (p < 0.05) or borderline significance (0.05 < p < 0.1) across all item clusters, with students who perceived irrelevance having a more novice-like shift in each case.

IV. DISCUSSION
While most students do perceive at least some relevance within a physics course towards their respective majors, a non-trivial percentage of students believe the course is irrelevant. Students who perceive irrelevance can have a significant effect on evaluating CLASS results from student populations. Relevance vs. irrelevance perception appears to have a moderate, statistically significant effect on attitudinal shifts for the first-semester algebra-based introductory physics course population that is predominately biology and health science majors. The robust effect appears to be sustained when looking at biology majors and (to a somewhat lesser extent) for health science majors.
The issue of relevance also appears to be important as a co-factor alongside learning goal orientation within the CLASS data. Students who are performanceoriented in particular appear more likely to have a novice-like shift if they perceive that the course is irrelevant to their choice of major. Students who are mastery-oriented (i.e., Framework-oriented in this context) do not appear to rely upon perceiving relevance, as they appear genuinely interested in mastering the material and appear to have an expert-like shift on average regardless. Conversely, a lack of a learning goal (i.e., Vague-oriented students) also seems to not depend on perception of relevance.
The lack of differentiation between students in this population on FCI pre-post gains, as observed in Ref. 8, persists with regard to students' perceived relevance vs. irrelevance of physics with respect to their majors. It appears that neither learning goals nor relevance is a factor as far as Force Concept Inventory responses indicate within the scope of a single physics course.
This follow-up study has three limitations to consider for future research and implementations. First, the study is done in a primarily undergraduate institution, and the sampled course is taught by an undergraduate department. Physics departments in similar circumstances will benefit in recognizing whether or not biology majors and health science majors must have their own respective course sequences designed for them, as well as which specific majors (e.g., health administration, which is primarily a business major at the host institution) in fact require algebra-based physics generally speaking. Different institution types may also determine whether learning orientations and relevance are a factor in the effectiveness of pedagogical course implementations, e.g., with large IPLS courses that are monolithically populated by a single life science major. Second, the measurements of relevance and learning orientation were determined with post-test data to examine where students were at the end of the course, with learning goals orientation determined from student responses to a specific pre-lab exercise that was based on research in metacognitive problem solving techniques. This suggests a more general identification of learning goal orientations is necessary, beyond the scope of a specific pedagogical implementation. Third, a pretest measurement of learning goals and relevance may be necessary to explore whether they are static or growthrelated aspects of mindset [4,7], as student mindset may be shaped over the course of a semester if perceived relevance or irrelevance changes.