Exploring optimal learning moments at tutorial sessions

At an optimal learning moment (OLM) a student experiences balanced and high levels of challenge, interest and skill at the same time. The frequency of these experiences was recorded at university physics tutorial sessions using an adaptation of the experience sampling method. The method proved to be a practical way to collect data on students' feelings during physics learning. Students who experienced optimal learning moments more often got better scores in the final exam, which shows that pursuing OLMs is worthwhile. Frequency of OLMs also seems to be connected with having favorable attitudes towards physics. These initial results imply that measuring OLMs could be used as a tool to recognize and support effective physics learning.


I. INTRODUCTION
Teaching at the Department of Physics, Helsinki University, Finland has been reformed thoroughly over the last couple of years. Interactive lecture methods, activelearning tutorial sessions and open-ended lab activities have led to more positive attitudes, highly improved retention rates and, in some cases also better learning. To further improve learning, we decided to probe students' feelings during tutorial sessions, and investigate if and how those feelings support effective learning.
Numerous studies consider the effects of emotions on physics learning, and the connection between students' affective states and physics learning has been studied in light of the concepts of self-efficacy [1] and flow [2] among others. In this study, students' feelings are explored through the theoretical framework of optimal learning moments (OLM) [3].
The concept of OLM builds on the ideas of flow [4] and situational interest [5] and is defined as a situation where a student feels high levels of challenge, interest and skill at the same time. Challenge pushes students forward to pursue new skills and it has been demonstrated to enhance students' engagement [6]. In light of flow-theory and OLMs, challenge should be balanced with the feeling of skill to avoid frustration. Feelings of skill and self-efficacy are also connected with success in physics studies [7]. The third component of an OLM, interest, is seen to be one of the most important factors in engagement and persistence in pursuing new skills [8]. Situational interest that is directed towards the specific activity is especially important for OLMs in contrast to more permanent personal interest towards physics in general. It has also been suggested that moments of optimal learning are associated with other subjective feelings, such as happiness and confidence, which may further enhance learning [3,9].
Experiencing an OLM during learning means that a fine balance between the student's skills and other qualities, teacher's actions (choice of suitable exercise, building motivation etc.) and the situation in general (subject matter, learning environment) has been reached. Measuring the frequency of students' OLMs during physics learning might therefore serve as a tool to determine the effectiveness of the learning situation. Determining the frequency and distribution of OLMs among students helps the teacher to adjust the level of challenge and support they provide and/or recognize the need to motivate students further. From the physics education research point of view it is interesting to find out how OLMs relate to more standard measures of physics learning efficiency and see how knowledge of students' feelings connects with the existing research on student learning.
As an initial step to determine the utility of the concept OLM in physics education, we studied how often physics students experience an OLM during tutorial sessions and how OLM frequencies correlate with conceptual and attitudinal survey data. We also looked into the connection between OLMs and final exam performance. While the concept of OLM builds on the simultaneous presence of high challenge, interest and skill, it was also interesting to investigate how the three prerequisites correlate with test and exam results. These steps are expected to show how measuring OLMs could be utilized in the development of physics teaching, and to give guidelines for future work. Further aspects, such as the relation of OLMs to students' background information and students' actions at the moment of measurement, will be covered in subsequent papers.

II. CONTEXT AND SAMPLE
The participants of this study were first-year physics students at the University of Helsinki. To get a place at the department the students need to qualify roughly among the best quarter in physics and mathematics exams in the Finnish matriculation exam. The matriculation exam is typically taken after three years of high school, during which the candidates take at least eight courses of physics and twelve courses of mathematics (one course equals 30 hours of instruction). The broad high school curriculum and fairly strict selection to Helsinki university ensures that entering students are well motivated and have relatively good knowledge in physics (for example FCI [11,12] pre-test scores are 80 % correct on average). There is considerable variation among students' skills, however, as some of the entrants are among the nation's most talented youngsters while others have barely fulfilled the entrance requirements.
This study was undertaken during the spring season of the first year. A calculus-based course in electromagnetism was under way. The course covers chapters 14-21 of the Matter & Interactions textbook [13] (with some additional topics) and consists of lectures, tutorial sessions and laboratory activities.
Tutorial sessions run for six hours each week, a four-hour session right after the lectures and a two-hour session two days later. In the tutorial sessions students solve problems in small groups (or alone, if they wish, but only a few do), with well-prepared teaching assistants guiding them. Groups can set the pace by themselves and there are no collective activities during the sessions. Typically all groups, regardless of their skill level, work in more or less the same manner, solving problems one at a time and also finishing their solutions before moving on. This is to say that even though the individual challenges vary a lot, all the students seem to have adopted a similar way of studying at the tutorial sessions. Tutorial sessions are voluntary, and students can utilize them as much as they like. The majority of course participants take part in the tutorial sessions quite actively, supposedly because exercise problems are scored and they make up one third of the course total.
Exercise problems emphasize problem solving and involve lots of calculations while lectures concentrate more on learning concepts. Although reformed, the electromagnetism course is typically perceived by the students to be quite mathematical and some students have trouble grasping the concepts behind the calculations. This culminates in the exercise problems. At the time of the study about 90 students were actively studying on the course (which is also the total number of first-year physics students at the University of Helsinki). Participants were either physics majors (85 %) or maths/chemistry majors with physics as a secondary subject. A total of 85 students took part in the tutorial sessions during this study. Of these, 65 students filled in the answer form at least once, handing in 252 answers altogether. Taking part in the study was fully voluntary, and some students chose to stay out. The participants do not necessarily represent all first-year physics students: it is not uncommon, especially for the most skilled students of our department, to skip tutorial sessions. In any case the participants represent well enough those who take part in tutorial sessions in the first place.

III. RESEARCH METHOD
Moments of high challenge, interest and skill are temporary and subject to change depending on the prevailing activity. That is why it is important to record students' feelings while they are working, rather than afterwards. One of the strengths of the OLM framework is that a practical and validated experimental tool for determining OLMs is readily available. The Experience Sampling Method (ESM) [10], which records students' feelings right in the middle of whatever they are doing, has been used in flow-studies for decades.
Typically ESM data is gathered with a short questionnaire. The participants are asked to fill out the form randomly throughout the day and this is repeated for a period of time. Collecting data throughout the day was now abandoned as we focused solely on the tutorial sessions. Data was collected once an hour during the six hours of weekly tutorial sessions. This was done twice, four weeks apart. We adapted the standard ESM self-report form [14] to suit this study. The self-report form consisted of statements for which answers were collected on a likert type scale. Three of the statements are relevant to this paper: 1. Were you interested in what you were doing? 2. Did you feel skilled at what you were doing? 3. Did you feel challenged by what you were doing? We also asked whether students were working on exercise problems, some other tasks or doing something unrelated to studies. Some other information was collected as well, for which the analysis was still underway while this paper was being written. Permission to combine given answers with final exam and test results was also requested. It took students two minutes on average to fill in the form.
In practice the researcher went to the tutorial session semirandomly once an hour, and asked the students to fill in the electronic self-report form. (Physics students are given a laptop at the beginning of the studies, so this is not a problem; some students also used their smartphones or other devices). In preparation for the data collection a short briefing was given at the corresponding week's lectures, where the background of the study and directions on how to answer were presented. If someone had missed that, the same information was given at the tutorial session individually.
OLMs were operationalized by creating a binary variable as in the reference study [3]. If a student reported four or more on a six-point likert-type scale, this was interpreted to mean that he/she felt high challenge, skill or interest. If all three factors were reported at a score of four or more at the same time, he/she was considered to be having an OLM. A binary variable was considered adequate for the purpose of finding the overall levels of OLMs, but for finding the correlations between factors of OLM and other measures an average without dichotomic recoding was used.
For this study, the shortened version of the test [16] with three subcategories (Personal Application and Relation to RealWorld (PARRW), Problem Solving/Learning (PS/L) and Effort/Sense Making (E/SM)) was used. A conceptual survey on electromagnetism (CSEM) [17] was adiministered together with the CLASS test as a pre-test and as a post test in between the two data collection points. Final exam scores were acquired from the course administration. These results were treated as percentages of the maximum score. A normalized gain (<g>) [12] for CSEM was also determined.
The aforementioned results were correlated with OLM data. Spearman's rank correlation coefficient ( ) was used, because the OLM-variable is not purely an interval one and some non-normalities appeared in the score distributions.

IV. RESULTS AND DISCUSSION
Of all answers 45 % were given during an OLM. In 76 % of cases students felt high challenge, in 68 % they felt high skill and in 76 % they felt high interest. A graphical representation of the simultaneous presence of these feelings is given in Fig.  1. There is no relevant reference point available yet at this stage, but the OLM frequency appears gratifyingly high. The subcategories' scores in the attitudinal test were 65 % (PARRW), 45 % (PS/L) and 77 % (E/SM). The average scores in CSEM pre-test and post-test were 50 % and 64 % respectively. Average score in the final exam of the course was 50 %. Test and exam results are close to the long-term averages recorded at the department, which indicates that the sample is quite representative. Spearman's rank correlations between these variables are given in Table I.

A. OLMs' relation to standardized test results
The individual OLM frequency correlates significantly with the PARRW-subcategory of CLASS. PARRW probes how interested in physics students are and how valuable physics is to them. The connection between PARRW and high interest is therefore evident. The connecting thread between personal interest (measured by CLASS) and m o m e n t s o f e f f e c t i v e l e a r n i n g ( m e a s u r e d b y O L M questionnaire) might be an affection for physics that makes the student enjoy any physics-related activities. Having a love affair with physics could suppress negative feelings and give rise to high self-confidence, which in turn might lead to a more positive picture of one's own skill level or even seeing the challenge posed by physics exercises in a more positive and constructive light. Problem solving and learning skills (PS/L) and persistence to study (E/SM) do not correlate with experiencing OLMs. Connections between OLMs and conceptual knowledge levels are also nonexistent and OLM frequency varies independently of CSEM gain. This probably reflects the fact that students tend not to connect tutorial sessions with conceptual learning. However, there is a correlation between course exam results and OLMs. It might simply be due to the resemblance of the course exam to the weekly exercise problem sets: you learn what you practice. Thus, as long as the final exam measures relevant skills, pursuing OLMs during tutorial sessions seems well worth while.

B. OLMs factors' relations to test results
Students' experience of challenge during tutorial sessions is not significantly connected to any of the measures, which is an expected result (the connection is more likely to be found by looking at the difficulty of the given exercises, which will be done in the near future). Students' experiences of skill and interest are, by contrast, connected with conceptual pre-test scores and all subcategories of CLASS. This leads to the presumption that some of the teaching methods that are designed to support conceptual learning and/or favorable attitudes should also pave the way for experiencing OLMs.
It also seems that good studying skills and persistence to study (PS/L and E/SM) go more or less together with personal interest. This was somewhat expected as well, but also raises questions of which comes first in each situation. Unfortunately the dynamics of these connections are beyond the scope of this study.

V. CONCLUSIONS AND FUTURE WORK
Based on this small-scale preparatory experiment, an adapted ESM method is feasible for collecting data on students' feelings during physics study. Initial results imply that experiencing OLMs at tutorial sessions has an impact on the final exam results. The exact mechanism behind OLMs' effect on exam performance is somewhat unclear, but results imply that studying in a optimal state helps to learn the things being taught, which makes OLMs worth pursuing in any case. Results also suggest that making OLMs more frequent could partly be achieved by the same means as supporting students' favorable attitudes towards physics. The OLM frequency seems to be partly connected to other, more established measures, such as CLASS, but it also holds such unique properties that OLM frequencies cannot be predicted by the results of attitudinal and/or conceptual tests. This, together with OLMs' correlation with exam results, warrants further studies into the utilization of OLM measurements.
The results so far are very much one-off and any general conclusions or implications for instruction cannot be given yet at this point. Some of the uncertainties will clear after the full data set has been analyzed. In particular, knowledge about what the students were doing at the moment of each OLM measurement will clarify dependencies between measurables. However, a much larger base of data and trials across varying student populations are needed to reliably identify connections between OLMs and measures of student skills and attitudes. After that has been established, measuring OLMs might serve as a tool to evaluate the efficiency of physics learning and diagnose problems in teaching and studying. __________________________________________________________