Comparing the outcomes of two different approaches to CEFR-based rating of students’ writing performances across two European countries
Holzknecht, F., Huhta, A., & Lamprianou, I. (2018). Comparing the outcomes of two different approaches to CEFR-based rating of students’ writing performances across two European countries. Assessing Writing, 37, 57-67. https://doi.org/10.1016/j.asw.2018.03.009
Published inAssessing Writing
© 2018 Elsevier Inc.
This study investigated to what extent two teams of experienced raters from different European countries (Finland and Austria), using their own CEFR-based rating scale (one holistic and one analytic), agreed on the CEFR level of students’ writing performances. Both teams rated one hundred performances written by Austrian secondary school students based on two tasks. The Finnish raters (N = 3) applied a holistic CEFR-linked rating scale consisting of verbatim CEFR descriptors developed in Finland, while the Austrian team (N = 6) used an analytic CEFR-linked rating scale consisting of four criteria developed in Austria. The ratings were analysed using the Rasch model. Although there were individual differences in rater severity among both teams of raters, a clear pattern emerged from the data: The Austrian raters were slightly more lenient than the Finnish raters. Although there was a statistically significant difference in rater severity between the two groups, the actual scope of disagreement was small. Thus, overall, the two teams agreed to a large extent on the CEFR levels of the participants. ...
ISSN Search the Publication Forum1075-2935
Publication in research information system
MetadataShow full item record
Showing items with similar title or keywords.
Developing Automated Feedback on Spoken Performance : Exploring the Functioning of Five Analytic Rating Scales Using Many-facet Rasch Measurement Zansen, Anna von; Huhta, Ari (Jyväskylän yliopisto, 2022)In this study, we used the Many-facet Rasch measurement (MFRM) to explore the quality of ratings as well as the functioning of five analytic rating scales developed for automated assessment of L2 speech. This study is part ...
"I could give up many things but not that" : teachers' and pupils' experiences of using the European Language Portfolio in assessment Peltola, Sonja (2015)Arviointi on olennainen osa opetusta ja oppimista kouluissa. Kielitaidon arviointiin vaikuttavat käsitykset kielitaidosta ja aikojen saatossa kielitaidon arviointi on muuttunut lähes mekaanisesta kielitaidon osa-alueiden ...
Assessment of Undergraduate Students' Learning Outcomes in Finland (Kappas!) Kleemola, Katri; Ursin, Jani; Silvennoinen, Kaisa; Nissinen, Kari; Hyytinen, Heidi; Toom, AuliThe Kappas! study investigated the level of Finnish undergraduate students’ generic skills, what factors are associated with the level of generic skills, and to what extent these skills develop during higher education ...
Upper secondary school students' perceptions of using mobile phone applications in the formal assessment of speaking skills Jääskeläinen, Katja (2020)Tutkimuksen aiheena on lukion oppilaiden käsitykset älypuhelinsovellusten käytöstä englannin suullisen kielitaidon arvioinnissa. Suullinen kielitaito, sen arviointi ja mobiilioppiminen (Mobile assisted language learning, ...
Second language comprehensibility and accentedness across oral proficiency levels : A comparison of two L1s Tergujeff, Elina (Elsevier BV, 2021)Second language comprehensibility and accentedness are highly complex phenomena, and numerous studies have been conducted to better understand these constructs. However, research has seldom addressed L2 comprehensibility ...