On site observation of KPG oral examiners : Implications for oral examiner training and evaluation

Abstract

The paper focuses on the development and main findings of an observation project carried out by the Research Centre for English Language Teaching Learning and Assessment (RCEL) of the Faculty of English Studies, University of Athens, within the context of the Greek state exams for foreign language proficiency, known as KPG exams. The project, which was launched in November 2005 and is ongoing, involved on site observation of oral examiners as a means of evaluating and monitoring the KPG oral exams for English; the project aimed at gathering information about, inter alia, the conditions of the oral exam administration, examiner discourse practices, examiner efficiency and conduct. The project involved the design of structured observation schemes used by trained observers during each exam administration (from November 2005 to May 2008). The observation schemes (the categories of which were refined after each exam administration based on the analysis of results) focused mainly on the discourse practices of examiners who assume the role of interlocutor in the KPG oral exams. More specifically, specific observation categories were gradually developed focusing on the changes examiners/interlocutors made to task rubrics and their interventions to candidates’ language output. The observation scheme findings have provided a wealth of data concerning the frequency and nature of examiner interventions and have contributed significantly to the refinement and development of the KPG oral examiner training programme.