Estimating Programming Exercise Difficulty using Performance Factors Analysis

Abstract

This Work in Progress Paper studies student and exercise modelling based on pass/fail log data gathered from an introductory programming course. Contemporary education capitalizes on the communications technology and remote study. This can create distance between the teacher and students and the resulting lack of awareness of the difficulties students encounter can lead to low student satisfaction, dropout and poor grades. In many cases, various technological solutions are used to collect individual exercise submissions, but there are little resources for indexing or modelling the exercises in depth. Exercise specific feedback from students may not be easily obtainable either. In the present study, we attempt to create student-exercises models solely on pass/fail log data by using statistical techniques. We conclude that such data is insufficient for student modelling, but that it can be used to credibly estimate the difficulty of programming exercises.