University Applicants’ Critical Thinking Skills: The Case of the Finnish Educational Sciences

ABSTRACT This study investigates the quality of the critical thinking skills of applicants (n = 77) seeking entry to the faculty of educational sciences in a Finnish university and how these skills are associated with the applicant’s age, previous higher education experience, and matriculation and entrance examination scores. The data consist of the applicants’ responses to problem-solving tasks and their matriculation and entrance examination scores. Critical thinking skills were measured with comparison and argumentation tasks. The results indicate that comparison of the texts and analysis of the arguments they contained were more difficult tasks than putting forward arguments both for and against of one’s personal standpoint. In addition, previous experience of higher education predicted participants’ comparison skills and their matriculation examination grades predicted their argumentation skills. The feasibility of using critical comparison tasks in the entrance examination tests is discussed.

education has been found to develop certain elements of students' critical thinking, that is, the ability to approach problems from various viewpoints (Kim, Sharma, Land, & Furlong, 2013) and to evaluate knowledge and make informed judgments (Wass, Harland, & Mercer, 2011). Similarly, Pascarella and Terenzini (2005) found that senior college students had moderately higher critical thinking skills than freshmen, while, in contrast, Arum and Roksa (2011) detected only a small increase in students' critical thinking skills after three semesters of college studies.
In similar vein, research results on the relationship between critical thinking skills and academic achievement in the university are contradictory. For example, higher critical thinking skills have been found to be associated with higher academic achievement in some studies (Badcock, Pattison, & Harris, 2010;Gadzella, Ginther, & Bryant, 1997;Stark, 2012;Stone, Davidson, Evans, & Hansen, 2001;Williams, Oliver, Allin, Winn, & Booher, 2003), but not all. For example, Lobb, Wilkin, McCaffrey III, Wilson, and Bentley (2006) found no predictive relation between university students' critical thinking skills and their first-year academic achievement.
From the perspective of university pedagogy, fostering the learning of critical thinking is of crucial importance from the outset of studies (Bovill, Bulley, & Morss, 2011) as university freshmen face an abundance of new information, conceptions, and theories that is often contradictory in nature. In order to comprehend and make sense of this conflicting knowledge, students need to apply critical thinking skills. However, little is known about the critical thinking skills of new students or university applicants. Knowledge about the level of applicants' critical thinking skills, in particular, is almost nonexistent, although it could provide important information from the point of view of developing university admission procedures, such as entrance examinations, and university teaching. Therefore, the purpose of the present study is to examine critical thinking skills of university applicants.

Interconnections of critical thinking and epistemic beliefs
The various definitions of critical thinking suggest, at least in the context of higher education, that critical thinking is linked to epistemic beliefs, that is, students' understanding of the nature of knowledge (Hofer, 2001). A recent study on Finnish third-year bio-science students' epistemological beliefs supports the idea that epistemic beliefs and critical thinking are interconnected, as the students applied critical thinking in order strengthen their thinking in general (Hyytinen, Holma, Toom, Shavelson, & Lindblom-Ylänne, 2014). There is also evidence that the more education adults have received, the more probable it is that they will perceive knowledge as complex and constantly in flux (Schommer, 1998). Further, this belief in the complexity and perpetual transformation of knowledge increases the individual's willingness to take multiple perspectives and alter their thinking (Schommer-Aikins & Hutter, 2002). In brief, well-developed epistemic beliefs are related to important elements of critical thinking, namely to the ability to see alternative viewpoints and to reflect on one's own thinking.
Owing to the epistemic aspect of critical thinking, it is not surprising that cognitive developmental models have been suggested as a method of gaining insight into the progression of critical thinking skills (Kuhn, 1999). According to Kuhn (1999), these models provide a roadmap of development from simple to more complex skills. Perry (1970) was one of the first researchers to investigate the intellectual development of university students. His model describes the development of assumptions about knowledge (epistemic beliefs) through three levels, starting from Dualistic (absolute) thinking, through Relativistic thinking, to Committed thinking (Perry, 1970).
Baxter Magolda (2004) applied Perry's method and found that the majority of college freshmen held absolute epistemic beliefs, while a minority held a transitional way of knowing, in which certain areas of knowing include relativistic elements. By their senior year, the majority of students had adopted a transitional way of knowing. More recently, Finnish first-year students of education showed a similar transition from dualistic to more relativistic views after only three weeks' participation in an innovative research methods course (Lahtinen & Pehkonen, 2013). In contrast, in another Finnish study, third year bio-science students held neither an absolutistic nor a relativistic epistemic viewpoint, but a fallibilist view of knowledge, in which they believed that it is possible to find measures for assessing and comparing conceptions, theories, and beliefs (Hyytinen et al., 2014). In addition to education, age and epistemic beliefs are also linked together. Older students rely less on experts or authorities as sources of knowledge than younger students, believe less in simple knowledge, and are more likely to believe that knowledge is uncertain (Bendixen, Schraw, & Dunkle, 1998;Chan, 2003;Walker et al., 2009).
It has been suggested that many of the models describing the development of epistemic beliefs since the pioneering work of Perry share a common feature, that is, thinking moves towards more integrative forms (Kallio, 2011). In this connection, King and Kitchener's (1994) Reflective Judgment Model is described in more detail below, first, because it illustrates the integration of uncertain elements in knowing and, second, because it is also the theoretical foundation for the model that we applied in our data analysis (described later in this paper). The Reflective Judgment Model describes the progression of epistemic beliefs, namely how an individual's assumptions about knowledge are related to the justifications that individual offers for his or her solutions to ill-structured problems (King & Kitchener, 2004). An ill-structured problem does not have a simple correct or certain solution (King & Kitchener, 1994), and hence requires the individual to consider the advantages and disadvantages of alternative possible solutions (Jonassen, 1997(Jonassen, , 2000. Epistemic beliefs are argued to be central to solving ill-structured problems, as they affect how individuals select and defend their solution options (King & Kitchener, 1994).
The Reflective Judgment Model comprises seven stages that are grouped into three broader levels: pre-reflective thinking (Stages 1-3), quasi-reflective thinking (Stages 4-5), and reflective thinking (Stages 6-7) (King & Kitchener, 2004, p. 6). First, in pre-reflective thinking, knowledge is understood to be certain and gained through one's own perceptions or from a specific authority; hence, all problems are treated with high degree of certainty and ill-and well-defined problems are treated similarly (King & Kitchener, 1994). This absolute approach to knowledge makes elementary critical thinking skills possible, because claims are not accepted as such, but they are evaluated by regard to this certain knowledge through cognitive operations (Kuhn, 1999).
Second, the quasi-reflective level is characterized by individuals' realization that, because ill-structured problems have numerous possible solutions, which can be justified by various types of evidence, knowledge is uncertain and therefore evidence plays a central role in the construction of knowledge (King & Kitchener, 1994. The recognition that there are various types of evidence and solutions to problems is also essential in critical thinking, as it involves the skills of interpreting and evaluating information from various sources (Fisher, 2007). Last, on the reflective level of thinking, the evaluation of information becomes more sophisticated as the person understands that knowledge is context-dependent and that its quality can be assessed by the information at hand, and hence some knowledge can be better justified than other knowledge (King & Kitchener, 1994. Similarly, determining the most reasonable approximation to the truth is the purpose of critical thinking (Ennis, 2008).
Use of the Reflective Judgment Model has been successfully extended to the assessment of college students' critical thinking in essay assignments (Wolcott & Lynch, 1997). Subsequently, Lynch and Wolcott (2001) applied the Reflective Judgment Model and the Dynamic Skill theory (Fischer & Bidell, 2006) in developing their Steps for Better Thinking Model for assessing ill-structured problem-solving performance, especially in relation to learning outcomes in higher education (Wolcott, 2006(Wolcott, , 2011. The Steps for Better Thinking Model consists of four hierarchical levels (steps) describing the progression of ill-structured problem solving skills (Wolcott, 2011, p. 76): In Step 1 (Identifying), the individual recognizes the absence of a single solution to a problem, and identifies the relevant and yet uncertain information that is needed to solve the problem.
Step 2 (Exploring) is characterized by the exploration of interpretations of and connections between pieces of information that are needed to solve the problem. Evidence is interpreted from multiple viewpoints to examine the assumptions and reasoning underlying these viewpoints. Next, in Step 3 (Prioritizing), the alternative solutions to the problem are prioritized and the soundest option is chosen by applying guidelines developed after a comprehensive analysis and evaluation of the most important factors related to the solution. Lastly, in Step 4 (Envisioning), the individual's problem-solving skills are integrated into an ongoing process that aims to generate and use information to develop a better solution to the problem. In consequence, the limitations of a chosen solution are acknowledged and explained.

Critical thinking and argumentation
The interconnected and developmental nature of critical thinking and epistemic beliefs is evident in argumentation, which can be defined as "the process of developing arguments, the exchange of views, the seeking and provision of good evidence to support claims and propositions" (Andrews, 2005, p. 110). Typically, providing good evidence requires that one evaluates various types of information, which in turn demonstrates reflective thinking and good problem-solving skills. Argumentation is particularly valuable in higher education, where it works as a mediating element connecting abstract and critical thinking on a more practical level, such as in argumentative writing (Andrews, 2010). Wu (2006) argues that undergraduate essay assignments often require students to analyze and evaluate content knowledge, to choose their position in debates in their discipline, and to argue accordingly. Hence, argumentation is a key requirement of essay writing by undergraduate students (Wingate, 2012). Previous research results also suggest that students with better argument analysis skills seem to produce better arguments and counterarguments in their essays (Preiss, Castillo, Flotts, & San Martino, 2013). In general, however, it has been demonstrated that university students have problems in achieving mastery in argumentation skills (Marttunen, 1997), and that they lack sufficient understanding of what argumentative essay writing requires (Wingate, 2012). Bensley (2011) argues that both lack of agreement on which specific skills should be included in critical thinking skills and vague definitions of these skills have hindered research on critical thinking. In light of Bensley's critique, comparison and argumentation skills are defined in this study as the key critical thinking skills. In focusing on these two aspects of critical thinking skills, we are aware that this leaves out other important elements of critical thinking, such as dispositions and attitudes (see Butler & Halpern, 2011) and reflective dimensions. However, the two foci of critical thinking selected in this study can be argued for, first, by noting that both argumentation and argumentative reasoning skills are at the core of critical thinking (Kuhn, 1991). Second, comparison skills have been specified as one of the concrete operationalizations of analytical skills (Facione, 1990;Kuhn, 2005). Moreover, Halonen (1995) explicitly names comparison and contrasting skills as key critical thinking skills alongside analysis and prediction skills. In addition, it has been suggested that novice higher education students lack both the will and the effort needed to compare set texts (Andrews, Torgerson, & Beng-Huat, 2010).

Critical thinking and academic achievement
An important argument for developing critical thinking assessments in university admissions is that measures of a broad range of domain-general abilities could serve as additional predictors of students' academic achievement, alongside aptitude tests and their high school grade point average (HSGPA) (Stemler, 2012). The predictive values of aptitude tests and HSGPA for academic achievement in universities have been well verified in the USA (e.g., Geiser & Santelices, 2007;Sackett, Kuncel, Arneson, Cooper, & Waters, 2009;Schmitt et al., 2009). In Finland, while universities can decide their student admission criteria independently, matriculation examination grades and/or entrance examinations are the most commonly used. However, studies on the predictive value of the Finnish university admissions criteria are scarce. The few existing studies suggest that matriculation and entrance examinations predict pace of study, at least in the social sciences and engineering (Häkkinen, 2004) and in medical school (Lindblom-Ylänne, Lonka, & Leskinen, 1996, 1999. The matriculation examination evaluates knowledge of the upper-secondary school curriculum and students have to take a minimum of four subjects (Finnish Matriculation Examination, 2014). The mother tongue test is mandatory to all, and consists of two parts. The first part aims to measure candidates' textual skills (e.g., analysis skills) and the second part thinking on a more general level. The remaining three tests have to be selected from the following subjects: the second national language, a foreign language, mathematics, and one from the general studies test battery in sciences and humanities (Finnish Matriculation Examination, 2014). Previous research results suggest that success in the matriculation examination is mostly explained by previous academic achievement and parental level of education (Häkkinen, Kirjavainen, & Uusitalo, 2003;Kirjavainen, 2012).
In Finland, some studies have focused on developing entrance examination tests that measure a broader range of cognitive skills. For example, in the 1990s, the examinations for entry to the University of Helsinki medical school introduced new essay tasks measuring synthesis writing and critical thinking (Lindblom-Ylänne et al., 1996, 1999. Nonetheless, matriculation exam grades and traditional multiple-choice science tests were the strongest predictors of academic achievement, even if the synthesis writing task predicted study pace and the critical thinking task predicted academic achievement (Lindblom-Ylänne et al., 1996, 1999. The Finnish government has urged universities to streamline admissions by making greater use of matriculation examinations as admissions criteria and by increased cooperation in setting entrance examinations (Ahola & Kokko, 2001). As a result, most Finnish educational faculties have started to use a jointly organized entrance examination. Applicants are required to read a book (about 180 pages) consisting of scholarly articles, and to answer multiple-choice questions designed to measure memorization, comprehension, and the ability to utilize knowledge drawn from the articles (Malinen, Väisänen, & Savolainen, 2012). However, the entrance examination measures, to some extent at least, the same abilities as the matriculation exam, as a positive correlation has been found between the initial points given on the basis of the applicants' matriculation examination grades and entrance examination scores (Räihä, 2010).
The previous research results, thus, suggest that investigation of the quality of educational science applicants' critical thinking skills and the associations these skills have with applicants' entrance examination scores and matriculation examination grades would have benefits for the further development of the entrance examination. It is these questions that we tackle in this study.

The purpose of the study and research questions
New university students encounter an abundance of conflicting information from the outset of their studies. Universities should, therefore, select students with a good ability to deal with information of this kind, that is, students who possess at least moderate critical thinking skills. In particular, focusing on the development of critical thinking in teacher training has been proposed to increase the use of critical thinking in the society as a whole (Williams, 2005). This further suggests that investigation of the critical thinking skills of applicants to teacher education programs, and to programs of educational sciences in general, is important. Also, Stemler (2012) has suggested that university admission procedures could benefit from the inclusion of a component assessing critical thinking. However, little is known about the quality of the critical thinking skills of university applicants and how critical thinking could best be taken into account in screening for university admission. In Finland, the recent introduction of a new kind of entrance examination in the educational sciences has raised the question of the relationship between the matriculation examination grade and entrance examination score and the possible associations of these measures with measures of critical thinking. Therefore, the purpose of this study is to clarify the associations between critical thinking skills of educational science applicants and the currently used admission criteria. Because previous studies suggest that epistemic beliefs become more sophisticated with age and are also affected by having a university education, we also investigate whether age or previous experience of university studies are related to these applicants' critical thinking skills.
The following three research questions (RQ) were addressed: RQ1: What is the level of educational science applicants' critical thinking skills with regard to comparison and argumentation skills? RQ2: How are applicants' matriculation examination score, entrance examination score, age, and prior higher education experience associated with their critical thinking skills? RQ3: Do applicants' matriculation examination score, entrance examination score, age, and prior higher education experience predict their critical thinking skills?

Participants
The target group of this study consists of persons who had applied to study education (including programs in teacher education, early childhood education, special education, adult education, and general educational sciences) in one Finnish university in the year 2012 and who could be reached by their email address from the applicant registry (n = 3,660). Our original plan was to integrate the data collection with the national entrance examination for admission to the educational sciences. However, for administrative reasons this was not possible, and thus we had to search for volunteers and construct a separate test for this group. Personal invitations to participate in the study were sent by email to the applicants after the entrance examination (high-stakes test). Because it was expected that recruiting participants after a demanding entrance examination would not be easy, a draw for a mobile phone was used as an additional incentive to participate in the study. After the second invitation, a small sample (n = 77), consisting of 67 (87%) women and 10 (13%) men, enrolled as participants in the study. These voluntary participants ranged in age from 18 to 47 (M = 23 SD = 6.3 years). Most of them (84%) had completed their upper-secondary school education, a few (8%) had a bachelor's degree, 4% had completed a master's degree, and 4% did not provide any information about their educational background. The majority of the participants (55%) had previously taken some higher education courses (e.g., courses at the Open University or the University of Applied Sciences); the remainder (45%) had no previous experience of higher education studies. The mean number of self-reported study credits accumulated by those who had previously participated in higher education studies was 62.4 (SD = 100.2). It was not possible to obtain information on how many of the participants were admitted to the university, but we assume that the data includes both admitted and rejected applicants.

Data collection
On a broad level, two main methods for measuring critical thinking can be distinguished. In addition to general critical thinking tests measuring a broad range of skills by using multiple choice items, another approach is to use essays or other forms of constructed measures (Butler & Halpern, 2011;Niu et al., 2013). In this study, the latter approach was selected as argumentative writing tasks, in particular, require the use of many critical thinking skills in the same task.
An online learning environment was used for the data collection. The invitation sent to the applicants included individual usernames and passwords for the online learning environment.
After having logged into the environment, the participants read an introduction page, which contained general information about the critical thinking tasks and the learning environment. Next, the participants read more specific instructions on how to use the environment, followed by a consent form. The participants were then asked to read three texts, each of which was an abridged version of a longer source text. All three abridged texts discussed class size and its effect on learning and academic achievement. The first source text was from the journal of the Finnish teachers' union, the second was a letter to the editor of a national newspaper from a professor of special education, and the third was from an article on the effects of class size on academic achievement published in a scholarly journal. The source texts were purposefully chosen to include partially contradictory information and conflicting viewpoints on the topic. The total length of the three abridged texts was 9 pages (2,819 words).
After reading these texts, the participants completed two ill-structured source-based writing tasks, designed to measure critical thinking by writing short essays. Task 1 (Comparison task) was: "Compare the views presented in the three texts on the effects of class size on learning and academic achievement." Task 2 (Argumentation task) was: "On the basis of what is said in the texts, present and justify your own viewpoint on whether class sizes should be reduced or not." After finishing these two tasks, the participants answered a short feedback questionnaire enquiring about their approach to learning and their experiences related to performing the tasks, and asking for some general background information.

Data of the study
The data consisted of the participants' writing task essays, their entrance examination scores, and their matriculation examination grades. The essays were written in Finnish and were saved automatically to a file in the online environment. The writing task data comprised 77 essays for task 1 (mean length 243 words, SD = 119) and 76 essays for task 2 (mean length 179 words, SD = 89). The participants' matriculation and entrance examination results were retrieved from the university's student services office.

Data analysis
A review of the research literature clearly demonstrated that critical thinking is entwined with epistemic beliefs. Consequently, developmental models of epistemic beliefs could be used to construct criteria for categorizing critical thinking skills into levels of quality. In this study, the Steps for Better Thinking model (Wolcott, 2011) was applied, both because it is theoretically grounded on such a developmental model and because it has also been previously applied in evaluating higher education students' critical thinking in several disciplines (e.g., Puteh & Hamid, 2014;Sungur, Tekkaya, & Geban, 2006;Zygmont & Karen, 2006). Argumentative essay writing tasks were also applied, as well-developed argumentation can be seen to require similarly developed epistemic beliefs. When analyzing the quality of the applicants' critical thinking skills, a whole essay was used as the unit of analysis. Four analytical categories (scored from 0 to 3) were used with all the critical thinking variables. The analytical framework for critical thinking skills was based on Wolcott's (2011) Steps for Better Thinking model, where 1 point corresponds to step 1, 2 points to step 2, and 3 points to step 3.

Comparison task
In the comparison task (Task 1), the participants were asked to compare the information and viewpoints presented in the various source texts. The following variables were used to analyze the quality of the participants' critical thinking skills: (1) Comparison of texts, (2) Conclusion based on the comparison, and (3) Number of viewpoints used in the comparison.
The variable Comparison of texts measures how well the viewpoints, opinions, and their justifications contained in the source texts were related to each other in the essay. It also measures how well similarities and/or differences between the texts were recognized and explicated. No points were given if information from the texts was presented in a disorganized manner or if the texts were not referred to at all; 1 point was given if information from the texts was merely described and presented one text at a time; 2 points were given if information and viewpoints from the different texts were compared; and 3 points were given if the information and viewpoints extracted from the texts were compared in relation to some broader perspective or context.
The variable Conclusion based on the comparison measured the quality of a summary included in the answer that was based on comparison of the texts. In the lowest category (0 points), the essay ended with no conclusion; 1 point was given if the essay included a conclusion indicating either some similarities or differences between the texts; 2 points were given if the essay included a conclusion in which different viewpoints drawn from the texts were identified, named, and related together; and 3 points were given if the essay included a conclusion in which the different viewpoints extracted from the texts were related together as a part of some broader context or perspective.
Last, the variable Number of viewpoints from the texts used in the comparison described how well the essay covered the topics relevant to the task. This quality was measured by counting how many of the viewpoints contained in the texts the participants used in their answers. In the lowest category (0 points), the answer included either no viewpoints or only 1 viewpoint; 1 point was given if either 2 or 3 viewpoints were used in the comparison; 2 points were given if the comparison included either 4 or 5 viewpoints; and for 3 points the comparison needed to include from 6 to 8 viewpoints. Examples of the analysis of the variables are given in Table 1. The comments in brackets are made by the authors.

Argumentation task
In the argumentation task (Task 2), the participants were asked to analyze and apply the information given in the three texts in presenting and justifying their own viewpoint. The quality of their argumentation was analyzed with the following variables: (4) Analysis of arguments in texts, (5) Arguments for and against, (6) Number of viewpoints used in argumentation.
The variable Analysis of arguments in texts focused on the examination of the arguments and their inter-relationships present in the texts (see Facione, 1990;Fisher, 2007). The focus was on how well a participant was able to name and state the relationship between the arguments included in the texts and whether this relationship was extended to some broader or new context. In the lowest category (0 points), either none of the arguments in the texts were analyzed or the analysis was unclear; 1 point was given if the arguments in the texts were only described and presented individually; 2 points were given if the relationship between the arguments present in the texts were made explicit; and 3 points were given if arguments from two or more text were analyzed as a part of some broader or new context.
The variable Arguments for and against measured how well the participants had put forward both arguments supporting their own standpoint, and counterarguments, and whether the argument and counterargument were integrated through rebuttal. Arguments are usually generated more in favor of a position a person supports than arguments countering it (Stein & Bernas, 1999). Expanding one's argumentation to include views counter to one's own demonstrates an understanding of alternatives, which is also considered an important element in high-level critical thinking (Fisher, 2007). Integrative rebuttal, which requires integration of an argument and counterargument by forming a rebuttal to the counterargument (Kuhn, 1991), is the most complex form of argumentation analyzed in this study. No points were given if no relevant arguments were presented in the answer; 1 point was given if only arguments supporting one's own standpoint were put forward; 2 points were given if the essay included both arguments supporting one's own stand and counterarguments against it; and 3 points were given if the essay included an argument for one's own standpoint, a counterargument against it, and also an integrative rebuttal connecting the argument and the counterargument. Successful integrative rebuttal is usually achieved in two principal ways: (1) by demonstrating a counterargument to be invalid and restoring the validity of the original argument, or (2) by comparing the original theory (argument) against the alternative theory (counterargument) and arguing that the original theory is the more valid one (Kuhn, 1991).
Number of viewpoints used in argumentation was determined by counting the number of the main viewpoints from the texts that the participant had used. The variable was scored in the same way as the variable Number of viewpoints used in the comparison task: in the lowest category (0 points) either none or only 1 viewpoint was used; 1 point was given if 2 to 3 viewpoints were used; 2 points were given if from 4 to 5 viewpoints were used; and 3 points were given if from 6 to 8 viewpoints were used. Examples of the scoring of the argumentation variables are given in Table 2.

Variable
Scoring example Interpretation Comparison of texts 0 points: S 70: "The extent to which the amount of pupils affects school success is probably school-specific. Especially in terms of how individual teaching can be arranged in the school. Everyone agrees that teachers are heavily burdened and they have too many irons in the fire all the time." Participant ignores comparison, indicated by the absence of information from the texts 1 point: S 1: "In the first text it came out that shy students suffer when compared to louder and more restless students. This occurs when the teacher's attention is taken up by these louder students, leaving these shyer students more in the background. … The second text went over a project which aimed to reduce class sizes [misunderstanding of the texts contents]. The association between class size and positive learning outcomes is not linear, but undulating. … The third text compared grade point averages between boys and girls in classes of different sizes … " In this excerpt, information from the texts is presented on the basis of individual texts, no comparison between the texts has been made.
2 points: S 48: "Teachers think that class sizes should be smaller, so that they could give enough attention to everyone and that everyone would get teaching that matches their own level and the attention they need. However, according to the results in the second text, [in fact, this result is from the third text] teachers wouldn't be able to change their teaching styles even if class sizes were reduced." Participant compares the content of text 1 about the benefits of smaller class size on teachers' teaching with the content of another text on the stability of teachers' teaching styles in classes varying in size.
3 points: S 98: "According to the texts by Saloviita [text 2] and Nissilä [text 1] both the highest educational administration and teachers see the reduction of class sizes as a significant approach to promoting the quality of teaching." Information from texts one and two are integrated under a broader concept (Quality of teaching) presented in text two.
Conclusion based on the comparison 1 point: S 88: "The three texts gave very different reasons for reducing class sizes. The first text dealt with this issue mostly through teachers' informal knowledge, without presenting any facts, whereas in the second text a professor of special education put forward his opinion. In the third text the research results were contradictory." Participant concludes that the texts gave very different reasons for reducing class sizes and describes those standpoints, but does not comment on the relationship between the texts.
2 points: S 2: "The studies clearly don't support smaller class sizes. However, teachers who work in these environments report experiencing smaller classes as better learning environments, and also as affecting their own work." In this excerpt the participant explicates the relationship between different perspectives about the effects of class size together with comparison of the research results and teachers' experiences.
3 points: S 12: "It can be concluded from all the three texts that small class sizes don't guarantee better academic achievement. Student misbehavior and the wellbeing of the teacher can be more easily influenced in smaller classes. Bigger schools and larger class sizes have more problems, but according to the research results, they don't seem to have an effect on students' grade point average." The effects of class size on academic achievement and problems in the classroom are integrated and followed by the conclusion that they do not seem to have a combined effect.

Variable Scoring example Interpretation
Analysis of arguments in texts 0 points: S 70: "I am a professional school assistant. It seems crazy how much worry is caused by teachers' workload and class sizes, while at the same time school assistants are suffering from unemployment or are working as substitute teachers." The participant ignores the information in the texts. Instead, she describes the situation from the point of view of her own occupation as a school assistant.
1 point: S 24: "It's difficult for a teacher to notice whether everyone has learned new things, if class sizes are enormous, as was stated in the first text. Individual teaching and attention given to students remains too limited." This excerpt illustrates how the arguments in the texts are listed but not analyzed.
2 points: S 11: "However, one clearly notices that even if students' success isn't badly disturbed in large-sized classes (rather the other way round), it is quite possible that teachers' ability to cope with the situation can be very limited." Participant compares the effects that large class sizes were claimed to have and analyzes arguments in the texts that make their relationship explicit.
3 points: S 99: "There are two noteworthy issues facing each other: teachers' coping and looking after students' wellbeing, and academic achievement. Which has greater importance? … As a teacher, the most important person for me in the classroom is the student. I want to support his/her education and learning. Even if recent Finnish research indicates that smaller class sizes have led to weaker academic achievement, I think that interaction between teacher and student plays a greater role in this." The participant integrates an argument about the class size effect on learning and an argument about interaction between a teacher and a student together under the broader concept of supporting a student's education and learning.
Arguments for and against 0 points: S 85: "The texts caused me to think quite contradictory thoughts. I was a pupil in a small rural school myself and at that time small class sizes were a commonplace thing. However, nowadays the amount of students in these so called school centers has prompted concern. Teachers don't have time to take the necessary precautions for the good of the children, as was noted in the first text." The participant has presented no arguments relevant to the task at hand.
1 point: S 6: "In my opinion class sizes should be reduced to a maximum of 20 students. I believe that reducing class sizes gives more benefits than disadvantages. … The teacher can take the quieter students and those with special needs better into account, when there are fewer students in the classroom." Only arguments supporting the participants' own standpoint are given.
2 points: S 98: "A survey conducted with teachers indicates that they do not see anything good in large class sizes, which can be expected. However, according to the research results presented in the texts the effect of class size is not so significant, especially if they are already reasonably sized." This excerpt shows how an argument that supports smaller class sizes (teachers didn't find anything good to say about them) and a counterargument against it (class size effects are not significant) were linked together.
3 points: S 54: "I believe that a class size of about 15-20 students would be ideal. I think that in a class this size a teacher could also give individual guidance to students and there could also be a couple of so called special needs students in the classroom. It was stated in the introduction to the third texts that teachers wouldn't change their teaching styles if class sizes were significantly reduced. I think that after the teacher has got used to this new class size and its new possibilities for herself, she would develop her teaching methods in a way that is more beneficial for the class." First, the participant gives an argument supporting his/ her own viewpoint (teachers would have more interaction with students), then counterarguments against it (teachers wouldn't change their teaching styles), and finally a rebuttal that integrates the argument and the counterarguments together (teachers would develop new teaching methods after they have got used to new class sizes).
In order to determine the reliability of the analysis, 15 essays from each task (20% of all the essays) were analyzed by two judges. Disagreements between the judges were discussed. The inter-rater reliabilities (Cohen's Kappa) for the variables of the comparison task varied from .81 to 1.0, and for the argumentation task from .70 to 1.0.

Formation of sum and background variables
Two sum variables were formed to describe the participants' overall performance in both the Comparison and the Argumentation task. The sum variables were formed by summing the values of the original three critical thinking variables for the comparison and argumentation tasks separately, and then by dividing the sum values by three to recreate the original scale (scale from 0 to 3). The sum variables were named as follows: Comparison, sum total (Cronbach's α .72) and Argumentation, sum total (Cronbach's α .80).
The variable age described the participant's age in years, and variable HE credits indicated the higher education course credits accumulated by the participants. The variables essay, 1 length and essay 2, length were formed by counting the number of words in the participants' essays.

Statistical analysis
Descriptive statistical analysis was used to describe the quality of critical thinking skills (RQ1), and bivariate statistical analysis (Pearson correlation coefficient) was used to examine the associations between the participants' critical thinking skills, matriculation and entrance examination scores, and background variables (RQ2). Furthermore, in order to test to what extent the participants' matriculation and entrance examinations, age, and higher education experience predicted their critical thinking skills (RQ3), multivariable regression analysis (linear regression) was used. Statistical analyses were conducted with IBM SPSS Statistics 20. The variables used in the statistical analyses are summarized in Table 3.

Level of the participants' critical thinking skills
The results showed that, on average, the applicants' performance was around the middle of the critical thinking skills scale (min 0 and max 3), as can be seen from the total mean scores (see Table 4) for the Comparison (1.43) and Argumentation (1.52) variables. Furthermore, their level of thinking in the use of the number of relevant viewpoints used in comparison in their essays was twice as high as their level in presenting conclusions on their comparison (1.90 versus 0.88). Likewise, the applicants' performance on the analysis of the texts' arguments was about half a level lower than their performance in presenting arguments both for and against their own standpoint (1.39 versus 1.96).
The results (Figure 1) showed that fewer than half of the participants were able effectively to compare the information in the texts (scored 2 points), and that about one-tenth extended their comparison into some broader context (scored 3 points). Conversely, more than half of the participants included no concluding summary in their essays (scored 0 points), and about one-tenth concluded their essay by describing only some features of the texts (scored 1 point). Further, about one-third of the participants explicitly drew attention to any relationship between two or three of the texts in their conclusion (scored 2 points), and only a few integrated the content of the texts into some broader context in their conclusions (scored 3 points). (1) Comparison of the texts (2) Conclusion based on the comparison (3) Number of viewpoints used in the comparison (4) Comparison, sum total (5) Analysis of arguments in texts (6) Arguments for and against (7) Number of viewpoints used in argumentation (8) Argumentation, sum total Descriptive statistics 2. How are applicants' matriculation and entrance examination scores, age, and prior higher education experience associated with their critical thinking skills?
(1) Comparison of the texts (2) Conclusion based on the comparison (3) Number of viewpoints used in the comparison (4) Comparison, sum total (5) Essay 1, length (6) Analysis of arguments in texts (7) Arguments for and against (8) Number of viewpoints used in argumentation (9) Argumentation, sum total (10) Essay 2, length (11) Matriculation exam, sum score (12) Entrance exam score (13) Age (14) Higher education credits Pearson correlation coefficient 3. Do applicants' matriculation and entrance examination scores, age, and prior higher education experience predict their critical thinking skills?
(1) Comparison, sum total (2) Argumentation, sum total (3) Matriculation exam, sum score (4) Entrance exam score (5) Higher education credits. In the Argumentation task, almost two-thirds of the participants inadequately analyzed the arguments in the texts, merely describing or completely ignoring arguments (scored 1 and 0 points) in their essays (Figure 2). However, the participants performed better when the task required the use of versatile argumentative perspectives, as almost three out of four successfully put forward both arguments for and counterarguments against their own viewpoint in their essays (scored 2 or 3 points). More importantly, a quarter of them reached the highest level, successfully integrating the argument and the counterargument through rebuttals.

Associations of critical thinking skills with matriculation and entrance examination scores and background variables
The correlations shown in Table 5 indicate that the better the matriculation exam grades and the higher the entrance examination scores a participant had, the higher also were his or her overall comparison skills. Likewise, the correlations indicated a similar positive association between the   .67** Note: In the statistical analysis, owing to missing background information, the numbers of participants were as follows: matriculation exam grades n = 67, entrance examination results n = 68, age n = 74, and higher education credits n = 72. *p < .05. **p < .01.
participants' matriculation and entrance examination scores and their overall argumentation skills: persons who performed better in both the matriculation and entrance examinations also possessed better argumentation skills. In general, the correlations presented in Table 5 indicate that argumentation skills (Variables 5 to 8) showed stronger associations with the matriculation exam grades and entrance examination test scores than comparison skills (Variables 1 to 4).
A closer investigation of the results for the comparison skills showed some low positive correlations between the variables of interest: the variable Entrance exam score correlated with the variable Comparison of texts (.26), and those applicants who succeeded better in the matriculation examination also had higher scores in the variable Conclusion based on the comparison (.24). The associations of the matriculation and entrance examinations with the skill to put forward arguments both for and against one's own standpoint were moderately positive (.34 and .40, respectively).
Age was not associated with any of the critical thinking skills variables. On the contrary, the more higher education experience the participant had, the higher were his or her comparison and argument analysis skills. Likewise, the length of the participants' response showed strong positive correlations with the points given for both the comparison sum total (.69) and argumentation sum total (.52). The greater the word count of the participants' essays was, the higher also were their critical thinking skills scores.

Matriculation examinations and entrance examinations and higher education experience as predictors of critical thinking skills
Multiple linear regression analysis was conducted to investigate whether success in the matriculation examinations and entrance examinations, or previous higher education experience predicted the participants' overall comparison and argumentation skills. Two separate analyses were conducted so that the variables Matriculation exam, sum score, Entrance exam score, and HE credits were entered as predictors, first for the variable Comparison, sum total and then for the variable Argumentation, sum total. Age was omitted from the analysis, as it did not show significant correlations with the dependent variables.

Discussion
Because critical thinking seems to be intertwined with epistemological beliefs (Hyytinen et al., 2014;Schommer-Aikins & Hutter, 2002), we first discuss the results of the quality of the critical thinking skills of our sample of applicants for entry to a university faculty of educational sciences in light of the Reflective Judgment Model (King & Kitchener, 1994) and other previous studies on the epistemological aspects of critical thinking. Typically, college students' epistemic beliefs can be located on the quasi-reflective level, in which multiple viewpoints and information related to ill-defined problems are recognized (King & Kitchener, 1994). In this study, most of the participants (in this study university applicants, not yet students) were able to identify the relevant viewpoints and arguments presented on various aspects of class size in the texts, which might be considered to reflect the quasireflective level of epistemic beliefs. Hence, the participants' performance could be interpreted to correspond to the relativist stance identified in Finnish first-year students in the educational sciences (Lahtinen & Pehkonen, 2013) or the quasi-reflective stance found among undergraduate students of accounting (Wolcott & Lynch, 1997). However, it must be borne in mind that the majority of the present participants were not completely unfamiliar with higher education and thus might have had previous experience in critical thinking tasks and essay writing.
Despite the view that epistemic beliefs and critical thinking are closely intertwined, the utilization of alternative viewpoints cannot be taken for granted. Undergraduate essay writing often requires analysis and evaluation of information (Wu, 2006). However, university students are claimed to lack both the motivation and effort to compare information when they write essays . Such a lack of comparison skills was also demonstrated in this study, as half of the participants merely described and listed the content of the texts or even ignored it. On the other hand, the other half of the participants successfully compared the texts, which runs rather counter to the findings by Andrews et al. (2010). The limitations noted in the participants' comparison skills could be explained by the task assignment, which did not explain how the comparison should be performed. Alternatively, the result could indicate limited knowledge of what an effective comparison requires, that is, the ability to recognize and point out the similarities and/or differences between texts discussing the same topic. Either way, the finding indicates that it might be beneficial to focus on teaching critical evaluation and comparison skills as early as in upper-secondary school as well as in the initial stages of higher education studies.
The quality of the participants' argumentation skills was also split in the same way as their comparison skills. Previous research has indicated that mastering argumentation skills is difficult for university students (Marttunen, 1997), and that low achieving students in argumentative essay writing have problems in being critical of and/or analyzing their source materials (Wingate, 2012). The present findings, showing that a large proportion of the participants were struggling in their analysis of the texts, are in line with these earlier results. In contrast to their argument analysis skills, one-half of the participants demonstrated at least an elementary understanding of argumentation (see Wingate, 2012), as they presented both arguments supporting their personal stand and counterarguments against it. In light of the fact that a quarter of the participants presented integrative rebuttals, the quality of their argumentation skills broadly resembles results that were previously reported among college students (Kuhn, 1991).

The associations of critical thinking skills with matriculation and entrance examination results and background variables
Earlier studies indicate that the more education one receives, the more sophisticated one's epistemological beliefs become (Baxter Magolda, 2004;Schommer, 1998), a pattern also echoed in the development of critical thinking (Arum & Roksa, 2011). The present results seem to support these findings, as the participants who had more higher education experience also had higher critical thinking skills. Similarly, our findings parallel earlier results, according to which critical thinking skills are positively associated with higher academic achievement (Badcock et al., 2010;Stone et al., 2001). The results indicated that success in both the matriculation and entrance examination was associated with higher critical thinking scores. These results differ from those reported for Finnish medical school entrance examinations, where tasks requiring critical thinking were not associated with matriculation examination grades (Lindblom-Ylänne et al., 1996).
Besides education, students' age has also been found to be associated with their epistemic beliefs, so that older students tend to believe less in authoritative sources of knowledge, and perceive knowledge to be more complex (Bendixen et al., 1998;Chan, 2003;Walker et al., 2009), which may also reflect better performance in critical thinking. Interestingly, in this study, age was not associated with the quality of the participants' critical thinking skills. However, there was rather a narrow variation in the participants' age, which may have caused the lack of effect of age. In addition, in this study, critical thinking skills were measured by using argumentative source based writing tasks, which measure skills that require academic training more than age-related skills. Therefore, these results need to be interpreted with caution.
Although matriculation and entrance examination outcomes and previous higher education experience were positively associated with the participants' critical thinking skills, these variables predicted their critical thinking skills inconsistently: the matriculation examination was the only significant predictor of an individual's argumentation skills, while previous higher education experience was the only significant predictor of an individual's comparison skills. These findings suggest that the Finnish matriculation examination is a valuable criterion in selecting students, and that university education has successfully developed students' critical thinking skills.

Implications of the study
Our results suggest that to some extent the matriculation examinations measure students' critical thinking, particularly argumentation skills. Furthermore, when taking into account the previous findings on the predictive validity of matriculation examinations for university studies (Häkkinen, 2004;Lindblom-Ylänne et al., 1996;1999), the demand by the Finnish Ministry of Education that the matriculation examination be more widely used as an admission criterion may be warranted (Ahola & Kokko, 2001;Ministry of Education, 2011).
On the other hand, it has been suggested that university admissions benefit from the use of broader cognitive measures alongside previous academic achievement (Stemler, 2012). Our results indicate that the critical comparison of information requires skills different from those evaluated in the matriculation examinations or in the entrance examination for degree programs in the educational sciences. Therefore, the possibility to include critical comparison tasks in, for example, the entrance examination used in the educational sciences, should be further examined. For instance, tasks requiring comparing and contrasting texts dealing with recently publicly debated topics in education could be suitable for this purpose. In previous studies, critical thinking skills have also predicted academic achievement (e.g., Gadzella et al., 1997;Lindblom-Ylänne, Lonka, & Leskinen, 1999;Stark, 2012;Williams et al., 2003), which further suggests that critical thinking skills should be evaluated more explicitly in entrance examinations.
From the perspective of university education, the applicability of the Steps for Better Thinking model (Wolcott, 2011), demonstrated in this study, suggests that it could be considered an assessment and teaching tool from the outset in study programs in the educational sciences. Coursework built on this kind of developmental model could help students to better understand how to properly carry out tasks requiring comparison and analysis of different viewpoints and arguments. University students' active engagement in learning critical thinking skills is important, especially since the development of these skills during university studies has been found to be mediocre or limited (Pascarella & Terenzini, 2005).

Limitations of the study
First, the number of participants (n = 77) was small, and therefore the sample cannot be considered representative of the population of applicants for entry into educational science faculties, which limits the generalizability of our findings. Moreover, linear regression analysis was used, despite the recommendation of a minimum sample size of 100 participants (Berk, 2004). However, as the other assumptions for regression analysis were met, we felt justified in using it, while acknowledging the limitations of having a small number of participants. Thus, the results of this study should be interpreted with caution.
Second, as the participants had to perform the tasks independently without training or live instructions on how to do so, they might have incurred additional cognitive load, thereby impeding their performance. However, the possibility of doing the tasks on a computer could also be less demanding on them, as writing with a computer enables flexible and versatile editing of answers.
Lastly, participants' prior knowledge or attitudes towards the topics that the texts discussed was not controlled, which could have had an effect on the results of the study.

Conclusion
Despite the limitations of this study, the results yield a valuable insight into an under-studied area: the quality of the critical thinking skills of educational science applicants. However, it is problematic to give any recommendations on whether the use of matriculation examination grades or alternatively the use of entrance examination tests should be emphasized. On the one hand, placing more weight on the matriculation examination grade would impede applicants who have not been successful in upper-secondary school. On the other hand, making more use of more entrance examination tests, which would measure critical thinking more explicitly, could benefit older applicants who might already have experience in higher education studies. To conclude, emphasizing one admission criterion at the expense of other possible criteria may favor certain applicants and discriminate against others. Therefore developing university admissions is a complex problem for which simple or straightforward recommendations cannot easily be issued.
For the development of admission criteria, it would be important in future studies to examine how educational science applicants' critical thinking skills are associated with success in being offered a place and how these skills predict their academic achievement and pace of study in the university. In order to develop and analyze the validity and reliability of the critical thinking measures, critical thinking could be investigated in relation to performance in standardized critical thinking tests. Finally, future studies exploring the quality of educational science applicants' critical thinking would benefit from the use of broader critical thinking measures, which would also include, for example, dispositions and attitudes towards critical thinking.