Uudenlaista kuullun ymmärtämistä – Kuvan ja videon merkitys ylioppilastutkinnon kielikokeissa

Abstract

This mixed methods research investigated the role of visual support such as pictures and video in assessing foreign language listening comprehension. Furthermore, this research gathered information on Finnish upper secondary school students’ beliefs about listening comprehension in order to reflect on the digitisation process of the Matriculation Examination which took place in 2016-2019 and the new curriculum that was implemented in the schools 2016. Test developers are starting to use multimodal material in high-stakes listening tests and therefore a better understanding of its role in foreign language assessment is needed. One hundred and fifty-seven learners of Swedish as a foreign language from different Finnish upper secondary schools participated in the study. The students in each group were randomly divided into three experimental conditions to compare their test performances and opinions. The students took a listening test that was the same for all conditions except for the amount of visual support they received: (1) audio-only (n=54), (2) pictures (n=52), and (3) video (n=51). The test included 31 items and was designed by the researcher for the B1-level according to the curricu-lum and following the current testing practices of the Matriculation Examination. The data consisted of students’ test performances (n=157) and their answers on a post-test survey (n=144), observations of the test situations (n=7), semi-structured interviews (n=4) and verbal protocols (n=3). The data were analysed using both clas-sical item analysis and item response theory analysis, other statistical approaches, as well as qualitative methods. The results showed no differences between the three experimental conditions on the level of the whole test (total raw score, total time, Rasch measure) but three statistically significant differences were found comparing the performances on item level. It is possible that in these items the audio-visual input had disturbed the listen-ing process in an unintended way. Students considered listening comprehension as a complex and useful skill needed especially in every-day communication. However, the teaching and assessment practices of listening comprehension used in the Finnish upper secondary schools appear to some extent outdated and archaic. According to the verbal protocols students processed test items differently. Overall, students pre-ferred video in listening comprehension tests. The benefits of video lie in its nonver-bal elements: seeing facial expressions, gestures and the context facilitates the com-prehension process. This study indicates that further research is required to better understand the role of nonverbal information in foreign language listening. This study encourages experts developing test items, learning material and curricula to take into account decisions they make concerning audiovisual material and possible impact of their products.