Modelling Perceived Segmentation of Bodily Gestures Induced by Music

Abstract

This article presents an ongoing investigation whose goal is to model perceived segmentation of music-induced bodily gestures. The investigation consists of three stages. The first stage is a database of multimodal recordings of people moving to music. The data of these recordings are video and motion-capture (acceleration and position at several points of the body). In the second stage the videos produced in the first stage are manually segmented. This is regarded as ground truth for the evaluation of the performance of an automatic gesture segmentation system developed in the third stage of the study. This system extracts kinetic features from motion-captured data. Then a novelty score is computed from the kinetic features. The peaks of the novelty score indicate segmentation boundaries. So far the kinetic features that have been evaluated are composed of only one windowed statistical function. None of them yields a reasonable similarity between computed and perceived boundaries. However, different functions of the kinetic features yield considerably similar results between perceived and computed boundaries at isolated regions of the data. This suggests that each of these functions performs best on a specific kind of gesture. Further work will consider evaluating kinetic features composed of combinations of functions.