METHODOLOGY DEVELOPMENT IN ADULT LEARNING RESEARCH COMBINING PHYSIOLOGICAL REACTIONS AND LEARNING EXPERIENCES IN SIMULATION-BASED LEARNING ENVIRONMENTS

In current theories of adult learning, there is lack of integrative, holistic framework that would integrate different elements of learning. Moreover, well-described methodological approaches for integrating different data types during naturalistic conditions are missing. We developed an interdisciplinary research design for multilevel approach on adult learning. We collected data from student - instructor dyads in aviation (n = 6 dyads) and forestry (n = 6 dyads) simulations using both quantitative (HRV, EEG; structured questionnaires) and qualitative (video recordings and stimulated recall interviews) data. We created a two-stage analysis protocol including modality specific pre-processing and quantification, and integrated multimodal analysis. First, the feasibility of achieving reliable physiological and neurophysiological recordings during learning experiences in naturalistic learning situations was established. Furthermore, a detailed description of subjective learning experiences was built on the basis of video-and interview data, including information on the challenges, emotions and student - instructor interaction during the learning situation. Two types of approaches were used for each modality: 1) an analysis based on structured pedagogical states and 2) an analysis based on continuous time-course of events during the learning situation. This developed methodological approach enables us to more comprehensively understand the factors that influence individual experiences and success of learning interaction. To develop pedagogical processes, it is necessary to build bridges also between the concepts and theories as well as between different disciplines like education and physiology. Our approach to adult learning enables new research lines that can integrate individual experiences, emotions, physiological and neurophysiological reactions during learning.


INTRODUCTION
Current theories of adult learning have been developed over decades.However, there is still a lack of an integrative, holistic framework that would include all different elements of learning phenomenon.Taylor ( [1]) wrote already twenty years ago that "the great challenge is to work toward finding common ground among many diverse but related theories of learning".He emphasizes that "we need to collaborate across disciplines, theories and paradigms to build a comprehensive theory of adult learning to guide educators of adults" ( [1]).
There are various influential theories of adult experiential learning ( [2], [3], [4]) aiming to conceptualize and explain how adult learning takes place and what is the role of experience in learning processes.For example, Mezirow highlights the meaning of critical reflection in his transformative learning theory (e.g.[3]), whereas Kolb has developed a cyclical model of experiential learning ( [2], [5]).Recently, Kolb has extended his theorizing towards a refined Learning Style Inventory (LSI) ( [6]).Schön's [4] theorizing particularly highlights the reflection-in-action as well as interaction between a learner and a coach in a professional context.He emphasizes that learning occurs in reflective practicum, i.e. in a dialogue between a learner and a coach.
Recently these theories have been criticized, mainly for two reasons (see e.g.[6], [7]).Firstly, there is still a lack of empirical evidence of the whole learning process and especially lack of interfacing with research findings of the physiological and neurobiological basis of learning.Thorough testing is needed in order to understand why these theories "work" in practice so successfully.Secondly, these theories are still partial or too narrow definitions of the complex and multifaceted phenomenon.For example, Mezirow's transformation theory relies most on the learner's rational and cognitive processes -emotions are in a peripheral role or "between the lines".There is an obvious need for more holistic theorization which takes into account both physiological and neurophysiological, emotional and reflective subprocesses of adult learning.
Learning situations can be approached as a subjective experience, or as a time-varying change in (neuro)physiological measures.Although these different aspects concern the same phenomenon of ongoing sequence of events, they have rarely been approached in an integrative manner, even at theoretical level.In his theory of consciousness, Antonio Damasio incorporates physiological reactions, brain mechanisms and subjective emotional experiences (i.e.feelings) ( [8], [9]).He also emphasizes the central role of (physiological) emotions on information processing, decision making and social cognition.These elements are a highly relevant part of the adult learning process.From the educational and psychological point of view, this means the same as highlighting the importance of emotions (and feelings) for learning.
Learning in a natural context cannot be understood by exploring one aspect in isolation, such as experienced feelings, physiological reactions (e.g.heart rate), cognitive elements of learning or studentinstructor interaction.With help of modern technology, we can measure physiological reactions during learning situations accurately even outside laboratories.Simulation environments provided by new technologies offer near authentic experiences.Further, they enable systematic variation in different components as well as a detailed study of their effects on learning.Accordingly, emergence of completely new research designs for approaching learning is enabled.However, it is important to recognize challenges in this kind of approach.All of the different methods and measurement modalities arise from different research traditions and even disciplines, which need to be acknowledged beyond simple technical solutions for processing heterogeneous data.In this paper, we describe our approach on the development of interdisciplinary holistic research design with methodological approaches that can integrate experiential and physiological measures of learning.

Combining learning experiences and physiology
During the past few years, there has been progress in the development of measures on how to make autonomic nervous system reactions visible (e.g.[10], [11]).Haataja et al. ([10]) described an association between monitoring behavior and physiological synchrony during collaborative learning in high school students, measured with wearable sensors and video-observations.Our previous study [12] suggested that physiological measures (heart rate) and learner-perceived experience are related.Physiological alertness as measured by changes in heart rate variability corresponded with the student's self-reported experiences.Both measures suggested that working in small groups facilitated by the teacher was physiologically more alerting/engaging and meaningful than the lectures.Vesisenaho et al. [13] reviewed the feasibility of studying learning experiences, specifically self-assessed experience of emotional involvement, in a virtual reality environment and of using (neuro)physiological measures in this context.
The emerging field of education neuroscience has in recent years started to explore multi-person situations and classroom interaction (e.g.[14], [15]).The focus in these studies is often in brain electrophysiology by electroencephalogram measurements (EEG), and specifically in the methods that can be used to approach inter-subject correlation ( [15]).However, the use of naturalistic settings in these studies offer useful methodological insights also for other types of research goals.Indeed, one of the core problems in interpreting (neuro)physiological measurements in learning situations, is the continuous nature of the measurements, as opposed to a usual experimental setting where sequences of single stimuli or tasks are used.There are no detectable features in the body physiology (e.g.heart rate based measures, electrodermal activity) and even less so in neurophysiology (EEG) that would directly link with a particular experience, yet alone learning.
However, there are some well-known variables that can be extracted from physiological measurements, and to some extent also from neurophysiological measures, that link with particular states, such as arousal, stress-level and vigilance [16], [17].When these measures are integrated with experiential and cognitive measures, the complementary information across modalities can give meaningful and novel information of opportunities and challenges in learning situation (see [13]).However, systematic study of the reliability and usefulness of such measurements needs yet to be shown.There are also studies (e.g.[18], [19]) in which experienced stress and heart rate have been measured during simulation-based learning (SBL) situations to determine the relationship between workload and self-perceived learning in the SBL situation.Girzadas et al. ( [18]), reported an increase in self-assessed learning with self-reported stress levels.Unfortunately, detailed content analysis of the learning experience as well as continuous monitoring of physiology throughout the learning situation is often lacking.

Creating holistic research design
In academic research, we often apply the approaches and methods from one discipline in one time.This is natural and often necessary when the research methodologies build on particular theoretical assumptions and frameworks.Focusing on single discipline and its methodological repertoire is not enough for understanding many of the real-life phenomena.Learning is a complex phenomenon that cannot be approached without integrating research methodology and even frameworks from different disciplines.Neuroscience methods can reveal the mechanisms and underlying brain processes that contribute to learning, but this information has a limited value for understanding learning situations, if it is not examined from the perspective of individual experiences.Likewise, qualitative interviews can give us an understanding of positive and negative experiences during learning, but the means to improve e.g.experienced stress and emotions are very shallow without deeper understanding of the bodily and brain processes involved.
While physiological recordings have already been applied in the study of classroom practice, at the conceptual level the study of pedagogical processes, learning experiences and (neuro)physiological phenomena are still far from each other.There is a lack of common vocabulary and framework which would be needed to promote scientifically rigorous research designs and, consequently, impactful research findings.The integration should be strongly supported by both educational and psychological theories, especially those concerning emotions and embodiment.

Simulationone of practices of experiential learning
Simulation-based learning (SBL) is a controlled learning situation, which is built through a certain pedagogical model and stages, and the instructor has a clear role in the process (see [2] for experiential learning).The structure of the SBL situation (Fig. 1) can be modified by either the learning context or technology (e.g. a virtual world).In addition, the structure can imitate an authentic environment and/or a task can be varied according to the instructor's instructions.Compared to traditional learning methods (e.g.lectures), simulations can be more powerful experiences due to authentic connection to the emotions and reflections they stimulate and which are also debriefed (e.g.[20], [21]).
In SBL situations, learning occurs in a fairly similar way as in real-life situations, and due to this feature it has been suggested as a powerful way of learning (e.g.[23], [24]).Furthermore, SBL offers an environment that can be optimally tuned for collecting both quantitative and qualitative data during an ongoing learning situation.

METHODOLOGY
From a methodological point of view, in order to utilize and integrate experiential data and physiological measures in the same experiment, it is necessary to find a compromise between several modality specific requirements.

General procedure
We created a multi-method approach that can be used to comprehensively explore brain electrophysiological activation, physiological reactions of the body, individual experience and student-instructor interaction in SBL situations.Data was collected in two learning situations (forestry and aviation).They followed a traditional SBL protocol consisting of the phases of introduction to simulation cases, action (simulations) and debriefing discussion (Fig. 1).

Figure 2. Student-instructor dyads in the SBL situations, forestry (left) and aviation (right) with EEG measurement devices.
Participants.A total of 12 male students (16-25 years) and 4 male instructors participated in the study in 2019-2020.Each instructor had three students under their guidance, resulting in 6 student -instructor dyads in aviation and 6 dyads in forestry (Fig. 2).In each setting, the instructor guided the student through exercises.The total duration of the data collection of each dyad lasted about 3,5 hours, including preparations for the measurements, the SBL situation (Fig. 1) and a stimulated recall interview.
The data collection methods included qualitative and quantitative measures.During the SBL situation we collected observational (video-recordings and written instructor observations), physiological (heart rate -based measures) and neurophysiological (electroencephalography [EEG]) data in a continuous manner.One student and one instructor dyad were measured simultaneously.After the actual SBL situation, stimulated recall interviews were performed with each student.In addition, HRV measures with an electronic diary were collected during a baseline period of 4-5 days.

Qualitative data
Video recordings.With the video recordings of the SBL situations, we have three-fold objectives.Firstly, to gather detailed event information on simulations enabling a further combination of both quantitative and qualitative data to the events by using timestamps.Secondly, the videos from SBL situations are utilised for analysing learning situations and carrying out behavioral analyses of studentinstructor interaction.Thirdly, video recordings are used as a material for stimulated recall interviews.
Instructor observations.The instructors filled their own forms during and after simulations while observing and instructing their students.The observations made by the instructors are utilised to supplement the information gathered from student's interviews.
Interviews.All students were individually interviewed after the SBL situations during the same or following day.The video recorded SBL sessions utilized a stimulated recall method both to stimulate the students' memory of events during the simulation but also to discuss the situation with the interviewer in detail.The students together with the interviewer watched the video recordings of the student's own SBL situation (the scenario).The students annotated their own videos with a special emphasis on pointing out episodes which were memorable and considered meaningful for their learning.These episodes were written into observation forms.The instructors were interviewed individually afterwards to explore their conceptions of the students' SBL situation and their own pedagogical thinking concerning SBL.Both student and instructor interviews lasted about an hour.

Quantitative data
Heart rate measurements.Heart rate -based variables can be used as an indicator for autonomic nervous system activity (e.g.[26], [27]).I Most typically used index, heart rate variability (HRV), is a measure of variation between successive heart beats.Intra-subject levels of stress and recovery can be identified ( [28]) by using algorithms that determine the involvement of parasympathetic and sympathetic nervous systems.We used two different HR measurement techniques: continuously during 3 to 5 consecutive days with a device with electrodes attached to the participants' chests ( [25]) and with EEG integrated ECG electrodes, placed beneath the collar bones, during the duration of SBL situation.We extracted both time-domain (mean, minimum and maximum HR, HRV), and frequency-domain measures (low frequency (LF) and high frequency (HF) components of HRV) from the HR(V) data, and utilize them as distinct measures (average over specified states) and as continuous measures (timecourse during the SBL and during longer baseline period).
EEG. EEG is a non-invasive physiological method to measure electrical activity of the brain with high temporal resolution.The method is based on recording the voltage differences between the electrodes on the surface of the skull.EEG can be used in naturalistic EEG recordings were carried out using Bittium NeurOne™ system.Signals from the student-instructor dyad were synchronously recorded using EEG caps with the exception of the forestry students who utilized the 13-channel EEG cap adapted for the VR headset.The 13-channel EEG cap was implemented with a Neoprene Headcap (Neuroelectrics®), NG Geltrode electrodes (Neuroelectrics®) and press stud cables.
Prior to a measurement, EEG caps were individually fitted to the heads of both the student and the instructor and the conductive gel was injected with a syringe into the holes of the electrodes.A ground electrode (neck), a reference electrode (mastoid), ECG electrodes (beneath the collar bones) and a respiratory belt were also attached.At the beginning of the recording a timestamp was added to the EEG data as an annotation and current time was displayed to the camcorder.This procedure was done to allow synchronizing the timing of the EEG data and the video data in the analysis phase.Additionally other annotations (e.g.beginning of the task) were included to the data during the measurement.

RESULTS
The following sections present the description of modality specific data analysis and the process of data structuring for the combination.Methodological development includes several steps in data processing, illustrated in Figure 3.The qualitative and quantitative data are first analysed independently, and structured in a way that enables integration of quantitative and qualitative data.This paper describes the protocol for modality specific and integrative analysis steps.

Modality specific analysis of qualitative data
The aim of the qualitative analysis is to structure and organize the research data in order to understand the course of the events during the learning situation as well as to explore the meaning of the SBL experience to the participants.
For the qualitative data, the interviews are first transcribed.Researcher triangulation of three education science researchers is used to read and analyse the transcribed interviews without preconceptions.The researchers are generating codes from the interview data by first independently interpreting the content and meaning of the interviewee's answers and comments.Second, the researchers compare the codes and interpretations they made with each other and, through a joint discussion, arrive at a common view of each sentence or comment of the interviewee.The qualitative analysis of the interview data emphasises the understanding of episodes that the participants considered meaningful for themselves.
Research approach is data-driven based on phenomenological and hermeneutical research traditions.
To enable the data combination of the qualitative and quantitative data, the data-driven codes generated through researcher triangulation are further used to build and select specific themes that can be placed in a temporal time continuum.Since the interview is conducted by following the simulation scenario steps chronologically and the discussion with the interviewee progresses temporally as the video is viewed according to the course of the simulation, it is possible to reconstruct the temporal time-line of student experiences while the SBL situation progressed.The timeline is created on the basis of the interview as well as the video and observation data connected to it.Temporal understanding of learning experiences is formed which contain e.g.information on the task variation, successes, failures and aspects of student-instructor interaction described by the students during SBL tasks.

Modality specific analysis of quantitative data
For the quantitative data, the focus is on extracting reliable artefact-free signatures that reflect the state and reactivity of the autonomic and central nervous system during different learning situations and task demands.The EEG data of each student-instructor dyad was split into separate files for each individual and for ECG and EEG data.Importantly, the data both across modalities (EEG, HR) and within the dyad (student, instructor) was fully synchronized, enabling later integration of both within and across subject time-courses.The EEG data was exported to Meggie software (GUI for MNE Python Analysis Software created in the Psychology Department of University of Jyväskylä) and analyzed in distinct episodes created using video-based annotations.In the preprocessing stage, electrodes with bad signal quality were removed, the influence of physiological artifacts (ocular and cardiac activity) was accounted for by independent component analysis (ICA) and the data was filtered to 1 -40 Hz.The data was downsampled from 1000 Hz to 333 Hz and re-referenced using the average of all EEG channels.The HR data was analyzed using Cubios Software ( [29]), which offers standard protocols for extracting the typical HR based measures used in the study (minimum and maximum of HR, HRV, Low and High frequency components of HRV).
After preprocessing of the data, two separate analysis pipelines were designed.First, we extract features specific to distinctive behavioral states (state-based analysis) during the ongoing SBL.Second, for a data-driven analysis we create time-courses of band-passed EEG signal following the continuous course of the SBL situation.These two analysis pipelines served different purposes.The state-based analysis enables us to focus on physiological and neurophysiological features in specific behavioral states (rest, easy task, difficult task, instruction, feedback).Both HR-based and EEG measures were examined in these distinct states, first individually in each participant and then averaged across participants.The time-course based analysis on the other hand is used to correlate ongoing brain activation with ongoing body physiology during the ongoing evolution of experience.Therefore, both analysis pipelines serve the ultimate goal of integrating the quantitative data with experiential, qualitative data using complementary analysis protocols.
IN EEG data, the focus was on rhythmic activation especially at the 10 Hz alpha band, which has been typically associated with arousal level, attention allocation and task engagement.10 Hz rhythmic activation is the most robust element in EEG and may thus be reliably extracted from the continuous EEG data in naturalistic condition.For extracting measures for the 10 Hz rhythm Fast Fourier Transform (FFT) algorithm was used to convert the preprocessed data from time domain to frequency domain.The power spectra of different states were calculated exploiting the annotating triggers.

Towards integrated interpretation of qualitative and quantitative data
In the next phase, the qualitative and quantitative data will be interpreted together to identify reproducible elements that reflect information for the understanding experiential learning and interaction.This analysis is performed separately for the student and the instructor, as well as for the interaction (synchrony measures) between the student and the instructor.The modality-specific analysis serves three functions: 1) To test the reliability and (across participants) reproducibility of measures collected during naturalistic continuous learning conditions.2) To extract features in each modality that are related to meaningful episodes during SBL. 3) To perform data transformation and structuring in a way that enables integration between modalities both within the quantitative measures (EEG, HR) and across the modalities (experiential, observational, physiological and neurophysiological measures).
Two different approaches for data structuring and transformation will be tested for the third (integrative) stage, i.e. for the mixed analysis of the data: First, the analysis based on structured pedagogical states, and second, the analysis based on a continuous time-course of events during the SBL situation.Of critical importance is that the EEG data and the video recordings during SBL were synchronized with the accuracy of a few seconds based on the timestamp annotation of the EEG data and the time shown on the video data.Thus, it is possible to put in parallel the different data sets (HRV, EEG, observational data and experiences and thoughts described in the interview) collected from the same SBL situation (student and instructor data).Based on the event analysis of the video recordings, the continuous SBL situation was divided into different states (e.g.rest, task and feedback).Figure 4 illustrates a timeline chart ('rag rug') created based on the timestamps from the annotations.Similar timeline of states was created for each student.This timeline provides an anchor for integrating the quantitative and qualitative data for the first, state-based analysis.The second analysis pipeline, based on the continuous time-course of events, represents a more datadriven approach.In this approach, the modality-specific data is transformed into a format that can be represented as a continuously varying trajectory.For quantitative data, this means a time-course of specific features extracted from HR and EEG data.For qualitative data, those thematic categories obtained through the analysis of the stimulated recall interview, that entail a temporal dimension, will be expressed as 'experiential time-course' during SBL.Further, the instructor observations and the videorecordings will be used to create a time-course of pedagogical interaction.These time-courses, representing changes in (neuro)physiological and experiential level, as well as in pedagogic interaction, will be used to pinpoint dependencies and causal association between events within and across individuals (dyad).

CONCLUSIONS
We explored whether and how physiological measurement technologies can be used in the combination with traditional educational research methods to explore subjective learning experiences.We collected data in aviation and forestry SBL situations.These settings enabled a controlled but still authentic exploration of learning experiences and student-instructor interaction.We described our approach on the development of interdisciplinary research design that can integrate experiential and (neuro)physiological measures of learning.We described a protocol for modality specific and integrative analysis steps in which qualitative and quantitative data are first analysed independently and thereafter structured and transformed to enable data integration.This kind of mixed methods approach requires collaboration across disciplines.Integration of experiential data and (neuro)physiological data necessitates compromises in modality-specific analysis protocols.Importantly, integrated interpretation requires generating novel approaches for data analysis.
Both the fields of neuroscience and education have clearly started to acknowledge the lack of power of the discipline-specific concepts and methods to approach the full spectrum of dimensions in the phenomenon of learning.Learning is both physiological and emotional with the cognitive elements being one part of the entity.It is typical that methodological possibilities appear before theoretical frameworks, either the present paper does not provide an interdisciplinary conceptual framework.However, to guarantee reliable interpretations of the contribution and predictive relationships between physiological states and the learning experience and interactions, it is important to develop ways for reliable data integration.Technological developments support this interdisciplinary collaboration since there has been a surge of new technologies to quantify physiological signals on-line.These are often integrated with algorithmic development to achieve easily usable and interpretable indexes.Commercial devices are available to record heart rate variability based stress-levels, sleep patterns, and even EEG for various reasons.
Although reliable information, also for research purposes, can be extracted from various signals in the human body and brain, the data analysis and interpretation remains superficial.Also in the field of education and pedagogy, (neuro)physiological measurement devices are sometimes applied without valid data processing procedures and rigorous interpretation of the findings.These types of 'pseudo neuroscience' can be even harmful for the emergence of truly useful integration of education and neuroscience research.
We aim to establish an understanding of the feasibility of achieving reliable physiological and neurophysiological correlates of learning experience.If feasible, this type of design enables us to more comprehensively understand the elements that influence individual experiences and success of learning interaction.It also enables us to explore the existence and meaning of synchrony between instructor and student in different measurement modalities.The novel approach in research methodology that we have developed in this project provides new opportunities to explore the empirical aspects of experiential learning more holistically than before.
Our teamwork across disciplines has shown an opportunity to develop new research lines that can approach learning simultaneously as experience, emotional states, and physiological reactions.We see this collaboration as the development of a common ground to explore holistic learning experience.Importantly, such a new perspective requires a new kind of theoretical integration.In the future, the development of theoretical integration should go alongside the methodological development.Specifically, physiological manifestations of experience could be seen as the part of the learning experience and thus as the part of the theoretical framework of experiential learning.