Automatic content analysis in collaborative inquiry-based learning

Abstract

In the field of science education, content analysis is a popular way to analyse collaborative inquiry-based learning (CIBL) processes. However, content analysis is time-consuming when conducted by humans. In this paper, we introduce an automatic content analysis method to identify the different inquiry-based learning (IBL) phases from authentic student face-to-face discussions. We illustrate the potential of automatic content analysis by comparing the results of manual content analysis (conducted by humans) and automatic content analysis (conducted by computers). Both the manual and automatic content analyses were based on manual transcriptions of 11 groups’ CIBL processes. Two researchers performed the manual content analysis, in which each utterance of the groups’ discussions was coded to an IBL phase. First, an algorithm was trained with some of the manually coded utterances to prepare the automatic content analysis. Second, the researchers tested the ability of the algorithm to automatically code the utterances that were not used in the training. The algorithm was a linear support vector machine (SVM) classifier. Since the input of the SVM must be a numerical vector of constant size, we used a topic model to build a feature vector representation for each utterance. The correspondence of the manual and automatic content analyses was 52.9%. The precision of the classifier varied from 49% to 68%, depending on the IBL phase. We discuss issues to consider in the future when improving automatic content analysis methods. We also highlight the potential benefits of automatic content analysis from the viewpoint of science teachers and science education researchers